Reproducibility Metrics Research Articles

Optimal Instruments for Measurement of Dietary Intake, Physical Activity, and Sleep Among Adults in Population-Based Studies: Report of a National Heart, Lung, and Blood Institute Workshop.

The National Heart, Lung, and Blood Institute convened a virtual workshop in September 2022 to discuss "Optimal Instruments for Measurement of Diet, Physical Activity, and Sleep." This report summarizes the proceedings, identifying current research gaps and future directions for measuring different lifestyle behaviors in adult population-based studies. Key discussions centered on integrating report-based methods, like questionnaires, with device-based assessments, including wearables and physiological measures such as biomarkers and omics to enhance self-reported metrics and better understand the underlying biologic mechanisms of chronic diseases. Emphasis was placed on the need for data harmonization, including the adoption of standard terminology, reproducible metrics, and accessible raw data, to enhance the analysis through artificial intelligence and machine learning techniques. The workshop highlighted the importance of standardizing procedures for integrated behavioral phenotypes using time-series data. These efforts aim to refine data accuracy and comparability across studies and populations, thereby advancing our understanding of lifestyle behaviors and their impact on chronic disease outcomes over the life course.

The stability of slow-wave sleep and EEG oscillations across two consecutive nights of laboratory polysomnography in cognitively normal older adults.

Laboratory polysomnography provides gold-standard measures of sleep physiology, but multi-night investigations are resource intensive. We assessed the night-to-night stability via reproducibility metrics for sleep macrostructure and electroencephalography oscillations in a group of cognitively normal adults attending two consecutive polysomnographies. Electroencephalographies were analysed using an automatic algorithm for detection of slow-wave activity, spindle and K-complex densities. Average differences between nights for sleep macrostructure, electroencephalography oscillations and sleep apnea severity were assessed, and test-retest reliability was determined using two-way intraclass correlations. Agreement was calculated using the smallest real differences between nights for all measures. Night 2 polysomnographies showed significantly greater time in bed, total sleep time (6.3 hr versus 6.8 hr, p < 0.001) and percentage of rapid eye movement sleep (17.5 versus 19.7, p < 0.001). Intraclass correlations were low for total sleep time, percentage of rapid eye movement sleep and sleep efficiency, moderate for percentage of slow-wave sleep and percentage of non-rapid eye movement 2 sleep, good for slow-wave activity and K-complex densities, and excellent for spindles and apnea-hypopnea index with hypopneas defined according to 4% oxygen desaturation criteria only. The smallest real difference values were proportionally high for most sleep macrostructure measures, indicating moderate agreement, and proportionally lower for most electroencephalography microstructure variables. Slow waves, K-complexes, spindles and apnea severity indices are highly reproducible across two consecutive nights of polysomnography. In contrast, sleep macrostructure measures all demonstrated poor reproducibility as indicated by low intraclass correlation values and moderate agreement. Although there were average differences in percentage of rapid eye movement sleep and total sleep time, these were numerically small and perhaps functionally or clinically less significant. One night of in-laboratory polysomnography is enough to provide stable, reproducible estimates of an individual's sleep concerning measures of slow-wave activity, spindles, K-complex densities and apnea severity.

Abstract 5016: Building better contrived samples to properly evaluate Next-generation MRD and LBx assays

Abstract Circulating tumor DNA (ctDNA) is a component of cell-free DNA (cfDNA) that originates from tumor tissue in the plasma of cancer patients. Although ctDNA is a broadly applicable tumor biomarker, the absolute amount of ctDNA varies across indication and stage, with the level of ctDNA present reflective of tumor volume. As novel methods for the characterization of ctDNA develop, there is need to evaluate these assays with appropriate samples. But current commercial reference standards, generated from modified cell lines or oligonucleotides lack important features such as DNA methylation, fragmentation pattern and nucleosome positioning, whilst clinical trial samples lack the required volume to perform multiple tests and often lack proper consent. We have created a framework for generating contrived samples containing the biological features of clinical samples and the attributes of a reference standard. The plasma-in-plasma contrived samples are generated from commercially purchased and fully consented matched tumor and plasma from cancer patients with age and gender matched healthy donor plasma. Samples are characterized by an internally validated 700 gene NGS panel to identify somatic variants and copy number alterations, which establishes ‘ground truth’. A contrived plasma dilution range based on the mean variant allele frequency (VAF) of the concordant somatic variants identified is generated. Using these methods, we have been able to generate sample sets for pan-cancer and disease specific evaluations. We have tested these samples through many novel high sensitivity assays that utilize both tumor-informed and tumor agnostic methods. We have confirmed that plasma-in-plasma contrived samples are amenable to all evaluated assays to date. The FFPE tumor tissue provide results matching clinical trial samples and the extraction yields of cfDNA are within the range of plasma samples from clinical trial data. Each dilution can be generated with replicates for reproducibility metrics. Current models of the sample dilutions have been generated to increase the number of replicates at relevant levels of tumor burden (i.e., 0.01% VAF). Negative samples consisting of the background plasma were also generated for each sample used. These plasma-in-plasma contrived samples have greater commutability to clinical samples than other commercial reference standards, however there are limitations. The complexity of generating these samples restrict the number of unique patients that can be utilized. Also, due to the volume of healthy plasma used as diluent, it is difficult to generate volumes greater than 4ml. To our knowledge, this is the first report examining the generation and usage of a contrived sample set that can be applied to evaluating novel high sensitivity MRD assays with clinical sample commutability. Citation Format: Paul Labrousse, Hugh Russell, David Shera, Daniel Stetson, Barrett Nuttall, Brian Dougherty, Darren Hodgson, James Hadfield. Building better contrived samples to properly evaluate Next-generation MRD and LBx assays [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 5016.

Comparison of wild-type and high-risk PNPLA3 variants in a human biomimetic liver microphysiology system for metabolic dysfunction-associated steatotic liver disease precision therapy.

Metabolic dysfunction-associated steatotic liver disease (MASLD) is a worldwide health epidemic with a global occurrence of approximately 30%. The pathogenesis of MASLD is a complex, multisystem disorder driven by multiple factors, including genetics, lifestyle, and the environment. Patient heterogeneity presents challenges in developing MASLD therapeutics, creating patient cohorts for clinical trials, and optimizing therapeutic strategies for specific patient cohorts. Implementing pre-clinical experimental models for drug development creates a significant challenge as simple in vitro systems and animal models do not fully recapitulate critical steps in the pathogenesis and the complexity of MASLD progression. To address this, we implemented a precision medicine strategy that couples the use of our liver acinus microphysiology system (LAMPS) constructed with patient-derived primary cells. We investigated the MASLD-associated genetic variant patatin-like phospholipase domain-containing protein 3 (PNPLA3) rs738409 (I148M variant) in primary hepatocytes as it is associated with MASLD progression. We constructed the LAMPS with genotyped wild-type and variant PNPLA3 hepatocytes, together with key non-parenchymal cells, and quantified the reproducibility of the model. We altered media components to mimic blood chemistries, including insulin, glucose, free fatty acids, and immune-activating molecules to reflect normal fasting (NF), early metabolic syndrome (EMS), and late metabolic syndrome (LMS) conditions. Finally, we investigated the response to treatment with resmetirom, an approved drug for metabolic syndrome-associated steatohepatitis (MASH), the progressive form of MASLD. This study, using primary cells, serves as a benchmark for studies using "patient biomimetic twins" constructed with patient induced pluripotent stem cell (iPSC)-derived liver cells using a panel of reproducible metrics. We observed increased steatosis, immune activation, stellate cell activation, and secretion of pro-fibrotic markers in the PNPLA3 GG variant compared to the wild-type CC LAMPS, consistent with the clinical characterization of this variant. We also observed greater resmetirom efficacy in the PNPLA3 wild-type CC LAMPS compared to the GG variant in multiple MASLD metrics, including steatosis, stellate cell activation, and the secretion of pro-fibrotic markers. In conclusion, our study demonstrates the capability of the LAMPS platform for the development of MASLD precision therapeutics, enrichment of patient cohorts for clinical trials, and optimization of therapeutic strategies for patient subgroups with different clinical traits and disease stages.

Quantitative metrics of the LV trabeculated layer by cardiac CT and cardiac MRI in patients with suspected noncompaction cardiomyopathy.

To compare cardiac computed tomography (CCT) and cardiac magnetic resonance (CMR) for the quantitative assessment of the left ventricular (LV) trabeculated layer in patients with suspected noncompaction cardiomyopathy (NCCM). Subjects with LV excessive trabeculation who underwent both CMR and CCT imaging as part of the prospective international multicenter NONCOMPACT clinical study were included. For each subject, short-axis CCT and CMR slices were matched. Four quantitative metrics were estimated: 1D noncompacted-to-compacted ratio (NCC), trabecular-to-myocardial area ratio (TMA), trabecular-to-endocardial cavity area ratio (TCA), and trabecular-to-myocardial volume ratio (TMV). In 20 subjects, end-diastolic and mid-diastolic CCT images were compared for the quantification of the trabeculated layer. Relationships between the metrics were investigated using linear regression models and Bland-Altman analyses. Forty-eight subjects (49.9 ± 12.8years; 28 female) were included in this study. NCC was moderately correlated (r = 0.62), TMA and TMV were strongly correlated (r = 0.78 and 0.78), and TCA had excellent correlation (r = 0.92) between CMR and CCT, with an underestimation bias from CCT of 0.3 units, and 5.1, 4.8, and 5.4 percent-points for the 4 metrics, respectively. TMA, TCA, and TMV had excellent correlations (r = 0.93, 0.96, 0.94) and low biases (- 3.8, 0.8, - 3.8 percent-points) between the end-diastolic and mid-diastolic CCT images. TMA, TCA, and TMV metrics of the LV trabeculated layer in patients with suspected NCCM demonstrated high concordance between CCT and CMR images. TMA and TCA were highly reproducible and demonstrated minimal differences between mid-diastolic and end-diastolic CCT images. The results indicate similarity of CCT to CMR for quantifying the LV trabeculated layer, and the small differences in quantification between end-diastole and mid-diastole demonstrate the potential for quantifying the LV trabeculated layer from clinically performed coronary CT angiograms. • Data on cardiac CT for quantifying the left ventricular trabeculated layer are limited. • Cardiac CT yielded highly reproducible metrics of the left ventricular trabeculated layer that correlated well with metrics defined by cardiac MR. • Cardiac CT appears to be equivalent to cardiac MR for the quantification of the left ventricular trabeculated layer.

Enhancing a Real-Time Flash Flood Predictive Accuracy Approach for the Development of Early Warning Systems: Hydrological Ensemble Hindcasts and Parameterizations

This study marks a significant step toward the future development of river discharges forecasted in real time for flash flood early warning system (EWS) disaster prevention frameworks in the Chugoku region of Japan, and presumably worldwide. To reduce the disaster impacts with EWSs, accurate integrated hydrometeorological real-time models for predicting extreme river water levels and discharges are needed, but they are not satisfactorily accurate due to large uncertainties. This study evaluates two calibration methods with 7 and 5 parameters using the hydrological Cell Distributed Runoff Model version 3.1.1 (CDRM), calibrated by the University of Arizona’s Shuffled Complex Evolution optimization method (SCE-UA). We hypothesize that the proposed ensemble hydrological parameter calibration approach can forecast similar future events in real time. This approach was applied to seven major rivers in the region to obtain hindcasts of the river discharges during the Heavy Rainfall Event of July 2018 (HRE18). This study introduces a new historical extreme rainfall event classification selection methodology that enables ensemble-averaged validation results of all river discharges. The reproducibility metrics obtained for all rivers cumulatively are extremely high, with Nash–Sutcliffe efficiency values of 0.98. This shows that the proposed approach enables accurate predictions of the river discharges for the HRE18 and, similarly, real-time forecasts for future extreme rainfall-induced events in the Japanese region. Although our methodology can be directly reapplied only in regions where observed rainfall data are readily available, we suggest that our approach can analogously be applied worldwide, which indicates a broad scientific contribution and multidisciplinary applications.

Reproducibility of cerebral blood flow, oxygen metabolism, and lactate and N-acetyl-aspartate concentrations measured using magnetic resonance imaging and spectroscopy.

In humans, resting cerebral perfusion, oxygen consumption and energy metabolism demonstrate large intersubject variation regardless of methodology. Whether a similar large variation is also present longitudinally in individual subjects is much less studied, but knowing the time variance in reproducibility is important when designing and interpreting longitudinal follow-up studies examining brain physiology. Therefore, we examined the reproducibility of cerebral blood flow (CBF), global cerebral metabolic rate of oxygen (CMRO2), global arteriovenous oxygen saturation difference (A-V.O2), and cerebral lactate and N-acetyl-aspartate (NAA) concentrations measured using magnetic resonance imaging (MRI) and spectroscopy (MRS) techniques through repeated measurements at 6h, 24h, 7days and several weeks after initial baseline measurements in young healthy adults (N = 26, 13 females, age range 18-35years). Using this setup, we calculated the correlation, limit of agreement (LoA) and within-subject coefficient of variation (CoVWS) between baseline values and the subsequent repeated measurements to examine the longitudinal variation in individual cerebral physiology. CBF and CMRO2 correlated significantly between baseline and all subsequent measurements. The strength of the correlations (R2) and reproducibility metrics (LoA and CoVWS) demonstrated the best reproducibility for the within-day measurements and generally declined with longer time between measurements. Cerebral lactate and NAA concentrations also correlated significantly for all measurements, except between baseline and the 7-day measurement for lactate. Similar to CBF and CMRO2, lactate and NAA demonstrated the best reproducibility for within-day repeated measurements. The gradual decline in reproducibility over time should be considered when designing and interpreting studies on brain physiology, for example, in the evaluation of treatment efficacy.

Single-cell Hi-C data Enhancement with Deep Residual and Generative Adversarial Networks.

The spatial genome organization of a eukaryotic cell is important for its function. The development of single-cell technologies for probing the three-dimensional (3D) genome conformation, especially single-cell chromosome conformation capture techniques (ScHi-C), has enabled us to understand genome function better than before. However, due to extreme sparsity and high noise associated with single-cell Hi-C data, it is still difficult to study genome structure and function using the HiC-data of one single cell. In this work, we developed a deep learning method ScHiCEDRN based on deep residual networks and generative adversarial networks for the imputation and enhancement of Hi-C data of a single cell. In terms of both image evaluation and Hi-C reproducibility metrics, ScHiCEDRN outperforms the four deep learning methods (DeepHiC, HiCPlus, HiCSR, and Loopenhance) on enhancing the raw single-cell Hi-C data of human and Drosophila. The experiments also show that it can generate single-cell Hi-C data more suitable for identifying topologically associating domain (TAD) boundaries and reconstructing 3D chromosome structures than the existing methods. Moreover, ScHiCEDRN's performance generalizes well across different single cells and cell types, and it can be applied to improving population Hi-C data. The source code of ScHiCEDRN is available at the GitHub repository: https://github.com/BioinfoMachineLearning/ScHiCEDRN. Supplementary data are available at Bioinformatics online.

The case of the disappearing librarians: analyzing documentation of librarians' contributions to systematic reviews.

The study aimed to analyze the documented role of a librarian in published systematic reviews and meta-analyses whose registered protocols mentioned librarian involvement. The intention was to identify how, or if, librarians' involvement was formally documented, how their contributions were described, and if there were any potential connections between this documentation and basic metrics of search reproducibility and quality. Reviews whose PROSPERO protocols were registered in 2017 and 2018 and that also specifically mentioned a librarian were analyzed for documentation of the librarian's involvement. Language describing the librarian and their involvement was gathered and coded, and additional information about the review, including search strategy details, was also collected. A total of 209 reviews were found and analyzed. Of these, 28% had a librarian co-author, 41% named a librarian in the acknowledgements section, and 78% mentioned the contribution of a librarian within the body of the review. However, mentions of a librarian within the review were often generic ("a librarian") and in 31% of all reviews analyzed no librarian was specified by name. In 9% of the reviews, there was no reference to a librarian found at all. Language about librarians' contributions usually only referenced their work with search strategy development. Reviews with librarian coauthors typically described the librarian's work in active voice centering the librarian, unlike reviews without librarian coauthors. Most reviews had reproducible search strategies that utilized subject headings and keywords, but some had flawed or missing strategies. Even among this set of reviews, where librarian involvement was specified at the protocol level, librarians' contributions were often described with minimal, or even no, language in the final published review. Much room for improvement appears to remain in terms of how librarians' work is documented.

Accuracy, repeatability, and reproducibility of T1 and T2 relaxation times measurement by 3D magnetic resonance fingerprinting with different dictionary resolutions

ObjectivesTo assess the accuracy, repeatability, and reproducibility of T1 and T2 relaxation time measurements by three-dimensional magnetic resonance fingerprinting (3D MRF) using various dictionary resolutions.MethodsThe ISMRM/NIST phantom was scanned daily for 10 days in two 3 T MR scanners using a 3D MRF sequence reconstructed using four dictionaries with varying step sizes and one dictionary with wider ranges. Thirty-nine healthy volunteers were enrolled: 20 subjects underwent whole-brain MRF scans in both scanners and the rest in one scanner. ROI/VOI analyses were performed on phantom and brain MRF maps. Accuracy, repeatability, and reproducibility metrics were calculated.ResultsIn the phantom study, all dictionaries showed high T1 linearity to the reference values (R2 > 0.99), repeatability (CV < 3%), and reproducibility (CV < 3%) with lower linearity (R2 > 0.98), repeatability (CV < 6%), and reproducibility (CV ≤ 4%) for T2 measurement. The volunteer study demonstrated high T1 reproducibility of within-subject CV (wCV) < 4% by all dictionaries with the same ranges, both in the brain parenchyma and CSF. Yet, reproducibility was moderate for T2 measurement (wCV < 8%). In CSF measurement, dictionaries with a smaller range showed a seemingly better reproducibility (T1, wCV 3%; T2, wCV 8%) than the much wider range dictionary (T1, wCV 5%; T2, wCV 13%). Truncated CSF relaxometry values were evident in smaller range dictionaries.ConclusionsThe accuracy, repeatability, and reproducibility of 3D MRF across various dictionary resolutions were high for T1 and moderate for T2 measurements. A lower-resolution dictionary with a well-defined range may be adequate, thus significantly reducing the computational load.Key Points• A lower-resolution dictionary with a well-defined range may be sufficient for 3D MRF reconstruction.• CSF relaxation times might be underestimated due to truncation by the upper dictionary range.• Dictionary with a higher upper range might be advisable, especially for CSF evaluation and elderly subjects whose perivascular spaces are more prominent.

Assessment of Regulatory Compliance Testing for Vitamin D in Infant Formula-Impact of Delegated Regulation (EU) 2019/828.

Since the publication of Standard Method Performance Requirements (SMPR®) for vitamin D in infant formula (SMPR 2011.004) by AOAC INTERNATIONAL, revised vitamin D limits have been recommended by the European Food Safety Authority (EFSA) for infant formula and adopted in Commission Delegated Regulation (EU) 2019/828. The vitamin D range introduced, 2-2.5 μg/100 kcal, is significantly narrower than previous limits specified by Codex Standard 72-1981 and requires lower method reproducibility metrics to adequately assess regulatory compliance. The narrower limits for vitamin D present a significant challenge for current-generation reference analytical methods that comply with SMPR 2011.004. We evaluate the impact of Delegated Regulation (EU) 2019/828 on the demonstrated performance of AOAC Method 2016.05/ISO 20636:2018 to assess the likelihood that vitamin D results produced by the method would be found outside the EU limits when testing infant formula that is compliant as manufactured. AOAC Method 2016.05/ISO 20636:2018, specifically data generated during multi-laboratory study, was used as a basis for statistical evaluation of the impact of the narrower EU vitamin D limits. The review of AOAC Method 2016.05/ISO 20636:2018 method performance against the vitamin D regulatory range introduced in (EU) 2019/828 indicates methods capable of performing in alignment with SMPR 2011.004 are likely to produce results that fail to meet EU requirements. Our assessment illustrates the high probability that a well-manufactured product with vitamin D levels within the EU regulatory range would fail to meet the regulatory requirements due to analytical method variability when tested using fit-for-purpose methods. Further, required method performance cannot be expected with the future development of new methods. To avoid this, consideration should be given to aligning proposed regulatory limits with method performance metrics of current-generation compendial methods. Current, state-of-the-art methods cannot consistently verify infant formula product compliance for vitamin D in accordance with (EU) 2019/828.

MO547: A Novel Quantitative Computer-Assisted Score can Improve Repeatability in the Estimate of Vascular Calcifications at the Abdominal Aorta

Abstract BACKGROUND AND AIMS Vascular calcifications (VCs), a known risk factor for cardiovascular disease and mortality, are strongly linked to vertebral fractures (VFs) [1]. The coronary calcium score from computer tomography (CT) is the gold standard to measure VCs [2], but this implies high radiation doses and costs. Although, several calcifications scores at the abdominal aorta have been developed from lower-dose and low-cost lateral spine RX projections. These scores are widely applied (the 0–24 score): Kauppila Score (KS) [3]. Most RX scores are semi-quantitative implying an error-prone visual assessment. Indeed, so far, repeatability studies could not estimate the minimum detectable difference (MDD). Based on this knowledge, we developed a new RX-based computer-assisted VCs score with the aim to test its precision (repeatability and reproducibility metrics and MDD estimate) compared to KS. METHOD New Quantitative Score (QS), computer-assisted tracking of calcifications, considers both abdominal aorta calcification and column length (range 0–1), then multiplied by 24 to easily compare with the standard score: KS. Dedicated software developed with ALBA framework (https://github.com/IOR-BIC/ALBA) to subsequently record standard and new scores (see Figure 1). We enrolled 44 patients with VCs from an earlier study and the anonymous recorded data were randomized in two series. Five experts from four specialties in two institutions evaluated the data twice, with 1-month interval between series. Additional test-retest evaluation was performed on eight subjects with a second RX within 2 weeks. RESULTS Intra-observer analysis (Bland-Altman plots and regression analysis between series): both scores seem not biased; QS showed higher reproducibility [95% confidence interval (95% CI) of differences almost halved]. Inter-observer [Intra Class Correlation Coefficient (ICC)]: ICC higher for QS in the first series (0.78 versus 0.64); a lower difference was observed in the second evaluation (0.84 versus 0.82). MDD (Standard Error of the Mean): reduced by around 25% for the QS (2.98 versus 4, in the 0–24 scale). Test-retest (Wilcoxon paired test): not significantly different for both scores. Agreement (Bland-Altman plots): scores are discordant. The QS shows significantly higher values and an increasing trend with calcification severity. CONCLUSION Our new quantitative computer-assisted score improved the precision of RX scores in the estimate of VCs. Furthermore, it showed higher intra-observer repeatability than KS and we reported for the first time an estimate of MDD in the VCs assessment. In detail, MDD was 25% lower for the new score. An ongoing study will determine whether this lower MDD may reduce follow-up times to check for VCs progression and whether this new VCs score may act as a risk alert for the development/worsening of VFs.

Quantification of green-blue ratios, impervious surface area and pace of urbanisation for sustainable management of urban lake – land zones in India -a case study from Bengaluru city

Quantification of the ecosystem services of blue and green infrastructure in urban centres with the perspective of planning sustainable development is usually data-intensive, includes use of multi-platform datasets and adds to the complexities in deriving effective and reproducible metrics. The present study describes the creation of four simple metrics to estimate: 1. the ratio of ‘green’ vegetated areas to the ‘blue’ water spread areas, defined as the ‘Green-blue ratio’ (GBA); 2. The ratio of ‘blue’ water spread areas to ‘built-up’ ratio around the lakes, defined as the ‘Blue to Built-up ratio’ (BBA), 3. the percentage of impervious surface area (ISA) and 4. the pace of urbanisation in the dynamic zones (DZ) of urban lake environments. These new metrics were evaluated using landcover areas mapped from satellite imageries. Visual interpretation-based method was adopted to delineate the green, blue and built-up areas from Google Earth, which is suitable for wide range of users. The use of these metrics has been illustrated using available datasets for four representative lakes in Bengaluru city, India: Sankey tank, Ulsoor lake, Nagavarakere and Puttenahallikere. Significant spatio-temporal variations in the ratios of GBA and BBA as well as %ISA were observed and satisfactorily reflected the ecological status of these lakes in concurrence with earlier studies. Detailed analyses constrained a permissible rate of annual increase in the built-up area within the DZs to ∼ 3% for sustainable development of the lakes. The present set of metrics can be recommended as useful tools for urban planners and citizen scientists for seasonal monitoring of urban lake environments.

HiCARN: resolution enhancement of Hi-C data using cascading residual networks.

MotivationHigh throughput chromosome conformation capture (Hi-C) contact matrices are used to predict 3D chromatin structures in eukaryotic cells. High-resolution Hi-C data are less available than low-resolution Hi-C data due to sequencing costs but provide greater insight into the intricate details of 3D chromatin structures such as enhancer–promoter interactions and sub-domains. To provide a cost-effective solution to high-resolution Hi-C data collection, deep learning models are used to predict high-resolution Hi-C matrices from existing low-resolution matrices across multiple cell types.ResultsHere, we present two Cascading Residual Networks called HiCARN-1 and HiCARN-2, a convolutional neural network and a generative adversarial network, that use a novel framework of cascading connections throughout the network for Hi-C contact matrix prediction from low-resolution data. Shown by image evaluation and Hi-C reproducibility metrics, both HiCARN models, overall, outperform state-of-the-art Hi-C resolution enhancement algorithms in predictive accuracy for both human and mouse 1/16, 1/32, 1/64 and 1/100 downsampled high-resolution Hi-C data. Also, validation by extracting topologically associating domains, chromosome 3D structure and chromatin loop predictions from the enhanced data shows that HiCARN can proficiently reconstruct biologically significant regions.Availability and implementationHiCARN can be accessed and utilized as an open-sourced software at: https://github.com/OluwadareLab/HiCARN and is also available as a containerized application that can be run on any platform.Supplementary information Supplementary data are available at Bioinformatics online.

MR Spectroscopy of the Insula: Within- and between-Session Reproducibility of MEGA-PRESS Measurements of GABA+ and Other Metabolites.

The insula plays a critical role in many neuropsychological disorders. Research investigating its neurochemistry with magnetic resonance spectroscopy (MRS) has been limited compared with cortical regions. Here, we investigate the within-session and between-session reproducibility of metabolite measurements in the insula on a 3T scanner. We measure N-acetylaspartate + N-acetylaspartylglutamate (tNAA), creatine + phosphocreatine (tCr), glycerophosphocholine + phosphocholine (tCho), myo-inositol (Ins), glutamate + glutamine (Glx), and γ-aminobutyric acid (GABA) in one cohort using a j-edited MEGA-PRESS sequence. We measure tNAA, tCr, tCho, Ins, and Glx in another cohort with a standard short-TE PRESS sequence as a reference for the reproducibility metrics. All participants were scanned 4 times identically: 2 back-to-back scans each day, on 2 days. Preprocessing was done using LCModel and Gannet. Reproducibility was determined using Pearson’s r, intraclass-correlation coefficients (ICC), coefficients of variation (CV%), and Bland–Altman plots. A MEGA-PRESS protocol requiring averaged results over two 6:45-min scans yielded reproducible GABA measurements (CV% = 7.15%). This averaging also yielded reproducibility metrics comparable to those from PRESS for the other metabolites. Voxel placement inconsistencies did not affect reproducibility, and no sex differences were found. The data suggest that MEGA-PRESS can reliably measure standard metabolites and GABA in the insula.

Translation of a circulating miRNA signature of melanoma into a solid tissue assay to improve diagnostic accuracy and precision.

Aim: Successful treatment of cutaneous melanoma depends on early and accurate diagnosis of clinically suspicious melanocytic skin lesions. Multiple international studies have described the challenge of providing accurate and reproducible histopathological assessments of melanocytic lesions, highlighting the need for new diagnostic tools including disease-specific biomarkers. Previously, a 38-miRNA signature (MEL38) was identified in melanoma patient plasma and validated as a novel biomarker. In this study, MEL38 expression in solid tissue biopsies representing the benign nevi to metastatic melanoma spectrum is examined. Patients& methods: Nanostring digital gene expression assessment of the MEL38 signature was performed on 308 formalin-fixed paraffin-embedded biopsies of nevi, melanoma insitu and invasive melanoma. Genomic data were interrogated using hierarchical clustering, univariateand multivariate statistical approaches. Classification scores computed from the MEL38 signature were analyzed for their association with demographic data and histopathology results, including MPATH-DX class, AJCC disease stage and tissue subtype. Results: The MEL38 score can stratify higher-risk melanomas (MPATH-Dx class V or more advanced) from lower-risk skin lesions (class I-IV) with an areaunder the curve of 0.97 (p<0.001). The genomic score ranges from 0 to 10 and is positively correlated with melanoma progression, with an intraclass correlation coefficient of 0.85 with stage 0-IV disease. Using an optimized classification threshold of ≥2.7 accurately identifies higher-risk melanomaswith 89% sensitivity and 94% specificity. Multivariate analysis showed the score to be a significant predictor of malignancy, independent of technical and clinical covariates. Application of the MEL38 signature to Spitz nevi reveals an intrasubtype profile, with elements in common to both nevi and melanoma. Conclusion: Melanoma-specific circulating miRNAs maintain their association with malignancy when measured in the hypothesized tissue of origin. The MEL38 signature is an accurate and reproducible metric of melanoma status, based on changes in miRNA expression that occur as the disease develops and spreads. Inclusion of the MEL38 score into routine practice would provide physicians with previously unavailable, personalized genomic information about their patient's skin lesions. Combining molecular biomarker data with conventional histopathology data may improve diagnostic accuracy, healthcare resource utilization and patient outcomes.

Deep Learning Improves the Temporal Reproducibility of Aortic Measurement.

Imaging-based measurements form the basis of surgical decision making in patients with aortic aneurysm. Unfortunately, manual measurement suffer from suboptimal temporal reproducibility, which can lead to delayed or unnecessary intervention. We tested the hypothesis that deep learning could improve upon the temporal reproducibility of CT angiography-derived thoracic aortic measurements in the setting of imperfect ground-truth training data. To this end, we trained a standard deep learning segmentation model from which measurements of aortic volume and diameter could be extracted. First, three blinded cardiothoracic radiologists visually confirmed non-inferiority of deep learning segmentation maps with respect to manual segmentation on a 50-patient hold-out test cohort, demonstrating a slight preference for the deep learning method (p < 1e-5). Next, reproducibility was assessed by evaluating measured change (coefficient of reproducibility and standard deviation) in volume and diameter values extracted from segmentation maps in patients for whom multiple scans were available and whose aortas had been deemed stable over time by visual assessment (n = 57 patients, 206 scans). Deep learning temporal reproducibility was superior for measures of both volume (p < 0.008) and diameter (p < 1e-5) and reproducibility metrics compared favorably with previously reported values of manual inter-rater variability. Our work motivates future efforts to apply deep learning to aortic evaluation.

The validity and reliability of ultra-short-term heart rate variability parameters and the influence of physiological covariates.

Ultra-short-term (UST) heart rate variability (HRV) metrics have increasingly been proposed as surrogates for short-term HRV metrics. However, the concurrent validity, within-day reliability, and between-day reliability of UST HRV have yet to be comprehensively documented. Thirty-six adults (18 males, age: 26 ± 5 yr, BMI: 24 ± 3 kg/m2) were recruited. Measures of HRV were quantified in a quiet-stance upright orthostatic position via three-lead electrocardiogram (ADInstruments, FE232 BioAmp). All short-term data recordings were 300 s in length and five UST time points (i.e., 30 s, 60 s, 120 s, 180 s, and 240 s) were extracted from the original 300-s recording. Bland-Altman plots with 95% limits of agreement, repeated measures ANOVA and two-tailed paired t tests demarcated differences between UST and short-term recordings. Linear regressions, coefficient of variation, intraclass correlation coefficients, and other tests examined the validity and reliability in both time- and frequency domains. No group differences were noted between all short-term and UST measures, for either time- (all P > 0.202) or frequency-domain metrics (all P > 0.086). A longer recording duration was associated with augmented validity and reliability, which was less impacted by confounding influences from physiological variables (e.g., respiration rate, carbon dioxide end-tidals, and blood pressure). Conclusively, heart rate, time-domain, and relative frequency-domain HRV metrics were acceptable with recordings greater or equal to 60 s, 240 s, and 300 s, respectively. Future studies employing UST HRV metrics should thoroughly understand the methodological requirements to obtain accurate results. Moreover, a conservative approach should be utilized regarding the minimum acceptable recording duration, which ensures valid/reliable HRV estimates are obtained.NEW & NOTEWORTHY A one size fits all methodological approach to quantify HRV metrics appears to be inappropriate, where study design considerations need to be conducted upon a variable-by-variable basis. The present results found 60 s (heart rate), 240 s (time-domain parameters), and 300 s (relative frequency-domain parameters) were required to obtain accurate and reproducible metrics. The lower validity/reliability of the ultra-short-term metrics was attributable to measurement error and/or confounding from extraneous physiological influences (i.e., respiratory and hemodynamic variables).

Reproducibility in machine learning for health research: Still a ways to go.

Machine learning for health must be reproducible to ensure reliable clinical use. We evaluated 511 scientific papers across several machine learning subfields and found that machine learning for health compared poorly to other areas regarding reproducibility metrics, such as dataset and code accessibility. We propose recommendations to address this problem.

Stenting "Vulnerable" But Fractional Flow Reserve-Negative Lesions: Potential Statistical Limitations of Ongoing and Future Trials.

Can imaging provide sufficient risk stratification to warrant revascularization of a stable plaque with negative fractional flow reserve (FFR)? Prophylactic stenting could at best be applied selectively since the composite group of FFR-negative lesions has a death or myocardial infarction rate of approximately 1%/year or less but modern stents have a rate of 2% to 3.5%/year. Because vulnerable features exist in a minority of lesions, at least 9,000 patients must be screened in order to enroll a cohort with sufficient risk. While several ongoing randomized trials are testing the concept of plaque sealing in FFR-negative lesions, preventive stenting depends on such a small effect that sample sizes to validate or refute its benefit become prohibitive. Since FFR provides a quantitative, straightforward, and reproducible metric of plaque vulnerability and burden without the need for or expense of additional catheter devices, intracoronary imaging cannot meaningfully guide prophylactic stenting when faced with a negative FFR.