Single Metric Research Articles

Lidar-derived estimates of forest structure in response to fire frequency

BackgroundLongleaf pine (Pinus palustris) ecosystems are recognized as biodiversity hotspots, and their sustainability is tightly coupled to a complex nexus of feedbacks between fire, composition, and structure. While previous research has demonstrated that frequent fire is often associated with higher levels of biodiversity, relationships between fire frequency and forest structure are more nuanced because structure can be difficult to measure and characterize. We expanded on this body of research by using lidar to characterize vegetation structure in response to fire frequency at a long-term prescribed-fire experiment. We asked (1) how does prescribed fire frequency affect structure and (2) how do structural metrics vary in the strength of their relationships with fire frequency.ResultsOur results indicated that forest structure varied significantly in response to fire frequency, with more frequent fire reducing vegetation structural complexity. Metrics that characterized the central tendency of vegetation and/or the variance of canopy-related properties were weakly to moderately correlated with prescribed fire frequency, while metrics that captured the vertical dispersion or variability of vegetation throughout the forest strata were moderately to strongly correlated with fire frequency. Of all the metrics evaluated, the understory complexity index had the strongest correlation with fire frequency and explained 88% of the structural variation in response to prescribed fire treatments.ConclusionsThe findings presented in this study highlight the usefulness of lidar technology for characterizing forest structure and that structural complexity cannot be fully characterized by a single metric. Instead, a range of diverse metrics is required to refine scientific understanding of the feedbacks between fire, composition, and structure in support of longleaf pine sustainability. Furthermore, there is a need for further research to broaden structural assessments beyond the overstory and incorporate more understory components, particularly within the realm of prescribed fire science and land management.

Global oceans suffer extreme heatwaves intensifying since the early 21st century: A new comprehensive index

As a result of global warming, major ocean basins have witnessed an increase in the number of extreme warm events and a decrease in the number of extreme cold events, increasing the number of marine heatwave (MHW) events. Previous quantification of MHW events has been limited to simple single metrics, which can only recognize some characteristics from a particular aspect. Here, we propose a new marine Heat Wave Comprehensive Index (HWCI) by fusing multiple metrics to characterize the scalable cumulative intensity of MHWs, which exhibits excellent identification reliability and superiority to effectively monitor the evolutionary patterns of various levels of MHW events. We find that five levels of global MHW events have presented an obvious spatial expansion and temporal enhancement pattern since the early 21st century, with the obvious spatial contraction (32.98 %) of weak events followed by the expansion (19.82 %) of extreme events at the highest growth rate of 0.07, primarily in the mid–low-latitude oceans and the Arctic. The results demonstrate that extreme MHW events dominate global MHW evolution patterns and that the expansion and intensification of such episodes have major implications for the event distribution and level structure. The new indicator is promising for directly measuring and identifying MHWs, and contributes to a more comprehensive understanding of the evolution of MHWs in the context of global climate change.

Exploring the Performance of Continuous-Time Dynamic Link Prediction Algorithms

Dynamic Link Prediction (DLP) addresses the prediction of future links in evolving networks. However, accurately portraying the performance of DLP algorithms poses challenges that might impede progress in the field. Importantly, common evaluation pipelines usually calculate ranking or binary classification metrics, where the scores of observed interactions (positives) are compared with those of randomly generated ones (negatives). However, a single metric is not sufficient to fully capture the differences between DLP algorithms, and is prone to overly optimistic performance evaluation. Instead, an in-depth evaluation should reflect performance variations across different nodes, edges, and time segments. In this work, we contribute tools to perform such a comprehensive evaluation. (1) We propose Birth–Death diagrams, a simple but powerful visualization technique that illustrates the effect of time-based train–test splitting on the difficulty of DLP on a given dataset. (2) We describe an exhaustive taxonomy of negative sampling methods that can be used at evaluation time. (3) We carry out an empirical study of the effect of the different negative sampling strategies. Our comparison between heuristics and state-of-the-art memory-based methods on various real-world datasets confirms a strong effect of using different negative sampling strategies on the test area under the curve (AUC). Moreover, we conduct a visual exploration of the prediction, with additional insights on which different types of errors are prominent over time.

Advancing Attribution-Based Neural Network Explainability through Relative Absolute Magnitude Layer-Wise Relevance Propagation and Multi-Component Evaluation

Recent advancement in deep-neural network performance led to the development of new state-of-the-art approaches in numerous areas. However, the black-box nature of neural networks often prohibits their use in areas where model explainability and model transparency are crucial. Over the years, researchers proposed many algorithms to aid neural network understanding and provide additional information to the human expert. One of the most popular methods being Layer-Wise Relevance Propagation (LRP). This method assigns local relevance based on the pixel-wise decomposition of nonlinear classifiers. With the rise of attribution method research, there has emerged a pressing need to assess and evaluate their performance. Numerous metrics have been proposed, each assessing an individual property of attribution methods such as faithfulness, robustness, or localization. Unfortunately, no single metric is deemed optimal for every case, and researchers often use several metrics to test the quality of the attribution maps. In this work, we address the shortcomings of the current LRP formulations and introduce a novel method for determining the relevance of input neurons through layer-wise relevance propagation. Furthermore, we apply this approach to the recently developed Vision Transformer architecture and evaluate its performance against existing methods on two image classification datasets, namely ImageNet and PascalVOC. Our results clearly demonstrate the advantage of our proposed method. Furthermore, we discuss the insufficiencies of current evaluation metrics for attribution-based explainability and propose a new evaluation metric that combines the notions of faithfulness, robustness, and contrastiveness. We utilize this new metric to evaluate the performance of various attribution-based methods. Our code is available at: https://github.com/davor10105/relative-absolute-magnitude-propagation

A Complexity Ladder for Big History

Complexity is a central problem for big history because big history has made complexity a central theme, constructing a cosmological periodization based on the sequential emergence of qualitatively distinct forms of complexity. How can the big historian differentiate distinct thresholds of emergent complexity while subordinating the entire sequence of thresholds to a single metric of complexity that demonstrates the increase of complexity over multiple scales of magnitude and across qualitatively distinct forms of complexity? The cosmologists’ use of a cosmic distance ladder suggests an analogous construction for complexity: a complexity ladder for big history. While no complexity ladder is formulated in this paper, the program required for a complexity ladder is sketched.

Evaluation of cardiac pro-arrhythmic risks using the artificial neural network with ToR-ORd in silico model output.

Torsades de pointes (TdP) is a type of ventricular arrhythmia that can lead to sudden cardiac death. Drug-induced TdP has been an important concern for researchers and international regulatory boards. The Comprehensive in vitro Proarrhythmia Assay (CiPA) initiative was proposed that integrates in vitro testing and computational models of cardiac ion channels and human cardiomyocyte cells to evaluate the proarrhythmic risk of drugs. The TdP risk classification performance using only a single TdP metric may require some improvements because of information limitations and the instability of generalizing results. This study evaluates the performance of TdP metrics from the in silico simulations of the Tomek-O'Hara Rudy (ToR-ORd) ventricular cell model for classifying the TdP risk of drugs. We utilized these metrics as an input to an artificial neural network (ANN)-based classifier. The ANN model was optimized through hyperparameter tuning using the grid search (GS) method to find the optimal model. The study outcomes show an area under the curve (AUC) value of 0.979 for the high-risk category, 0.791 for the intermediate-risk category, and 0.937 for the low-risk category. Therefore, this study successfully demonstrates the capability of the ToR-ORd ventricular cell model in classifying the TdP risk into three risk categories, providing new insights into TdP risk prediction methods.

Time-dependent complexity characterisation of activity patterns in patients with Chronic Fatigue Syndrome

BackgroundChronic Fatigue Syndrome patients suffer from symptoms that cannot be explained by a single underlying biological cause. It is sometimes claimed that these symptoms are a manifestation of a disrupted autonomic nervous system. Prior works studying this claim from the complex adaptive systems perspective, have observed a lower average complexity of physical activity patterns in chronic fatigue syndrome patients compared to healthy controls. To further study the robustness of such methods, we investigate the within-patient changes in complexity of activity over time. Furthermore, we explore how these changes might be related to changes in patient functioning.MethodsWe propose an extension of the allometric aggregation method, which characterises the complexity of a physiological signal by quantifying the evolution of its fractal dimension. We use it to investigate the temporal variations in within-patient complexity. To this end, physical activity patterns of 7 patients diagnosed with chronic fatigue syndrome were recorded over a period of 3 weeks. These recordings are accompanied by physicians’ judgements in terms of the patients’ weekly functioning.ResultsWe report significant within-patient variations in complexity over time. The obtained metrics are shown to depend on the range of timescales for which these are evaluated. We were unable to establish a consistent link between complexity and functioning on a week-by-week basis for the majority of the patients.ConclusionsThe considerable within-patient variations of the fractal dimension across scales and time force us to question the utility of previous studies that characterise long-term activity signals using a single static complexity metric. The complexity of a Chronic Fatigue Syndrome patient’s physical activity signal does not suffice to characterise their high-level functioning over time and has limited potential as an objective monitoring metric by itself.

Assessing topographic effects on forest responses to drought with multiple seasonal metrics from Sentinel-2

Topography determines run-off direction, redistributes groundwater, and affects land surface solar radiation loads and the associated evaporative forcing, consequently, topography can modulate the impact of drought and heat waves on ecosystems. This topographic modulation effect, which typically occurs at the local scale, is often overlooked when assessing ecosystem drought responses using moderate-to-coarse spatial resolution satellite observations, such as the Moderate-resolution Imaging Spectroradiometer (MODIS) imagery. Sentinel-2 and Landsat imagery with finer resolution are suitable for monitoring changes at the local scale, however, studies relying on single vegetation metrics may fail to get a holistic picture of vegetation drought responses, particularly for forests that have complex physiological mechanisms. Here, we performed a comprehensive assessment of the topographic effects on coniferous forest responses to the severe 2018 drought in Scandinavia, using 6 vegetation seasonal metrics during 2017–2021 from the Sentienl-2 High-Resolution Vegetation Phenology and Productivity (HR-VPP) products. We found significant differences (p < 0.05) between sunny and shady aspects, between higher and lower elevations, and between steep and gentle slopes, regarding the maximum impact time, forest drought resistance, and resilience. Specifically, the sunny aspects and steep slopes were related to higher risks of delayed impacts and low resistance and resilience, and elevation and slope were more powerful in regulating the phenology shift and greening rate loss. We also identified different sensitivity in greenness and productivity to topographic effect and greater sensitivity of spring phenology to topographic differences as compared to autumn phenology. The study demonstrates vegetation drought responses represented by multiple seasonal metrics, reveals the prevalent topographic effects at the local scale, and quantifies the magnitudes of the effects with regional statistics.

Classification Assessment Tool: A program to measure the uncertainty of classification models in terms of class-level metrics

Accuracy assessments are important steps of classifications and get higher relevance with the soar of machine and deep learning techniques. We provided a method for quick model evaluations with several options: calculate the class level accuracy metrics for as many models and classes as needed; calculate model stability using random subsets of the testing data. The outputs are single calculations, summaries of the repetitions, and/or all accuracy results per repetitions. Using the application, we demonstrated the possibilities of the function and analyzed the accuracies of three experiments. We found that some popular metrics, the binary Overall Accuracy, Sensitivity, Precision, and Specificity, as well as ROC curve, can provide false results when the true negative cases dominate. F1-score, Intersection over Union and the Matthews correlation coefficient were reliable in all experiments. Medians and interquartile ranges (IQR) of the repeated sampling from the testing dataset showed that IQR were small when a model was almost perfect or completely unacceptable; thus, IQR reflected the model stability, reproducibility. We found that there were no general, statistically justified relationship with the median and IQR, furthermore, correlations of accuracy metrics varied by experiments, too. Accordingly, a multi-metric evaluation is suggested instead of a single metric.

Trends in Soil Science over the Past Three Decades (1992–2022) Based on the Scientometric Analysis of 39 Soil Science Journals

As one of the basic disciplines of agricultural, natural resource, and environmental science, soil science has played a critical role in global food security and socio-economic and ecological sustainability. The number of soil science journals and publications has increased remarkably with the development of soil science. However, there is a lack of systematic and comprehensive studies on the developmental trends of soil science based on journals and publications. In this study, 39 journals included under the soil science category in the 2022 Journal Citation Reports, and 112,911 publications in these journals from 1992 to 2022 were subjected to scientometric/bibliometric analysis to determine trends in publication, journal metrics, co-authorship, and research topics, in addition to general journal information. The results show that soil science ushered in a renaissance period with the number of publications, citations, impact factors, and CiteScore demonstrating an increasing trend. America and the Chinese Academy of Sciences had the most publications and citations. The most productive author published more than 400 articles. Soil science research focused mostly on its fundamental impact on the ecological environment based on the strongest citation bursts analysis of keywords. The analysis indicated that open access has increased in popularity. Current soil science journals still face a few common challenges, including an urgent need for a fairer evaluation mechanism on journal quality compared to the traditional use of single metrics as well as equity, diversity, and inclusion (EDI) in the whole editorial process. Artificial intelligence may bring new tools and more changes to the development of soil science. This study will help soil science researchers to better understand the development status and future trends of soil science. It will also guide authors in journal selection.

Beyond a fixed number: Investigating uncertainty in popular evaluation metrics of ensemble flood modeling using bootstrapping analysis

AbstractEvaluation of the performance of flood models is a crucial step in the modeling process. Considering the limitations of single statistical metrics, such as uncertainty bounds, Nash Sutcliffe efficiency, Kling Gupta efficiency, and the coefficient of determination, which are widely used in the model evaluation, the inherent properties and sampling uncertainty in these metrics are demonstrated. A comprehensive evaluation is conducted using an ensemble of one‐dimensional Hydrologic Engineering Center's River Analysis System (HEC‐RAS) models, which account for the uncertainty associated with the channel roughness and upstream flow input, of six reaches located in Indiana and Texas of the United States. Specifically, the effects of different prior distributions of the uncertainty sources, multiple high‐flow scenarios, and various types of measurement errors in observations on the evaluation metrics are investigated using bootstrapping. Results show that the model performances based on the uniform and normal priors are comparable. The statistical distributions of all the evaluation metrics in this study are significantly different under different high‐flow scenarios, thus suggesting that the metrics should be treated as “random” variables due to both aleatory and epistemic uncertainties and conditioned on the specific flow periods of interest. Additionally, the white‐noise error in observations has the least impact on the metrics.

Hybrid board governance: Exploring the challenges in implementing social impact measurements

This paper focuses on hybrid board governance and the challenges faced by the board of directors when implementing social impact measurements. Interviews with 36 board chairs and general secretaries in social hybrids in Sweden show that while boards support social impact measurements, they face obstacles in implementing them. Drawing on the institutional logics framework, we identify three main reasons for these implementation problems. First, amid field level regulations focusing on cost efficiency, boards find it difficult to switch to a social impact that lacks a single metric that can be measured annually. Second, board members struggle to find sufficient time when they serve on a pro bono basis, and it is difficult to hold them accountable when limited progress occurs. Finally, acknowledging board practice variation, we highlight the need to distinguish between “beneficiary-driven” and “membership-driven” social hybrids. In the former, boards face the challenge of operationalizing the long-term benefits for end beneficiaries; in the latter, interactions with members are so operationally focused that boards struggle to maintain a long-term agenda for implementing social impact measurement. Given these challenges, we propose that future research should explicitly incorporate the board level in theorizing how hybrid organizations manage institutional logics and performance measurement.

Thermal limits of survival and reproduction depend on stress duration: A case study of Drosophila suzukii.

Studies of ectotherm responses to heat extremes often rely on assessing absolute critical limits for heat coma or death (CTmax), however, such single parameter metrics ignore the importance of stress exposure duration. Furthermore, population persistence may be affected at temperatures considerably below CTmax through decreased reproductive output. Here we investigate the relationship between tolerance duration and severity of heat stress across three ecologically relevant life-history traits (productivity, coma and mortality) using the global agricultural pest Drosophila suzukii. For the first time, we show that for sublethal reproductive traits, tolerance duration decreases exponentially with increasing temperature (R2 > 0.97), thereby extending the Thermal Death Time framework recently developed for mortality and coma. Using field micro-environmental temperatures, we show how thermal stress can lead to considerable reproductive loss at temperatures with limited heat mortality highlighting the importance of including limits to reproductive performance in ecological studies of heat stress vulnerability.

Predicting nodal influence via local iterative metrics

Nodal spreading influence is the capability of a node to activate the rest of the network when it is the seed of spreading. Combining nodal properties (centrality metrics) derived from local and global topological information respectively has been shown to better predict nodal influence than using a single metric. In this work, we investigate to what extent local and global topological information around a node contributes to the prediction of nodal influence and whether relatively local information is sufficient for the prediction. We show that by leveraging the iterative process used to derive a classical nodal centrality such as eigenvector centrality, we can define an iterative metric set that progressively incorporates more global information around the node. We propose to predict nodal influence using an iterative metric set that consists of an iterative metric from order 1 to K produced in an iterative process, encoding gradually more global information as K increases. Three iterative metrics are considered, which converge to three classical node centrality metrics, respectively. In various real-world networks and synthetic networks with community structures, we find that the prediction quality of each iterative based model converges to its optimal when the metric of relatively low orders (K∼4\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$K\\sim 4$$\\end{document}) are included and increases only marginally when further increasing K. This fast convergence of prediction quality with K is further explained by analyzing the correlation between the iterative metric and nodal influence, the convergence rate of each iterative process and network properties. The prediction quality of the best performing iterative metric set with K=4\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$K=4$$\\end{document} is comparable with the benchmark method that combines seven centrality metrics: their prediction quality ratio is within the range [91%,106%]\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$[91\\%,106\\%]$$\\end{document} across all three quality measures and networks. In two spatially embedded networks with an extremely large diameter, however, iterative metric of higher orders, thus a large K, is needed to achieve comparable prediction quality with the benchmark.

A Novel Approach for Routinely Assessing Laboratory Sigma Metrics for a Broad Range of Automated Assays.

Sigma metrics have been adapted for the clinical laboratory to incorporate observed accuracy, precision, and total error allowed. The higher the Sigma level for a process, the better performance that process has. A limitation of studies assessing Sigma metrics is that they are performed on a small number of well-controlled systems. An algorithm was developed to extract QC data and derive the Sigma metric for 115 analytes from sites connected to the QuidelOrtho E-Connectivity® database. The median of these results was then used to derive the Sigma metric for each assay. In this analysis, 79 out of 115 (68.7%) of the assays assessed achieved 6 Sigma or better and 98 out of 115 (85.2%) achieved 5 Sigma or better. This study has demonstrated a methodology that can be used to condense Sigma metrics from hundreds of analyzers into a single metric of assay quality. Because these analyzers are running in working laboratories from around the world, this analysis can serve as a baseline for understanding the assay performance achieved in the presence of variabilities such as lab-to-lab, instrument-to-instrument, material handling, environmental conditions, and reagent lot. The significant number of assays demonstrating high Sigma levels did so despite this variation. The ability of the methods reported here to include hundreds of analyzers represents a novel approach for assessing Sigma metrics in clinical laboratories.

The sensitivity of pressure-based knock threshold values to alternative fuels: A comparison of methanol vs. gasoline

The heavy-duty industry typically uses a high cetane fuel to power their vehicles, currently typically fossil diesel. Switching to renewable fuels such as methanol requires a major changeover in engine technology. Due to the high octane number of methanol, the diesel engines will have to be replaced by more suitable spark-ignited engines. An accurate knock control system will then be needed to accommodate the high power density requirements of this sector. However, current pressure-based knock detection methods were primarily developed for gasoline fuel operation. Blindly copying these methods for a fuel which has vastly different knock properties can lead to suboptimal engine operation with an efficiency and power loss as result. In this work, experiments were conducted on a single cylinder port fuel injected spark-ignited CFR engine. The applicability of five different pressure-based knock metrics was reassessed for methanol fuel and their performance was compared to each other. Knock threshold values were established for both gasoline and methanol for the MAPO (maximum amplitude of pressure oscillations), AE (average energy) and MVTD (minimum value of third derivative) methods. The results show that the performance of the single peak metrics MAPO and IMPO (integral of modulus of pressure oscillations), and the time-averaged metrics AE and HRR (heat release rate) remains identical between methanol and gasoline operation. For the MVTD method however, the suggested low sampling rate of 1 sample/°ca is insufficient for methanol combustion since the fuel by nature already shows a narrow pressure peak. Overall, the HRR method outperforms the others through the addition of a noise constraint. The calculated knock threshold values, however, are drastically different between both fuels. Compared to gasoline, the thresholds for methanol were 1.25, 2.40 and 5.45 times higher for the MAPO, AE and MVTD methods respectively. This highlights the need for dedicated, fuel-specific, knock threshold values.

Multipath phase based vegetation correction scheme for improved field-scale soil moisture retrieval over agricultural cropland using GNSS-IR technique

Global Navigation Satellite Systems-Interferometric Reflectometry (GNSS-IR) is an emerging remote sensing technique studied and demonstrated for soil moisture retrieval over bare soil using multipath GNSS data. However, retrieval of soil moisture in the presence of vegetation/crops is still a very challenging task even becoming a more complex problem without using any ancillary datasets to compensate the vegetation effect for soil moisture retrievals in higher crop growth stages. This paper presents a novel multipath phase-based vegetation correction scheme for improved field-scale soil moisture retrieval using L-band data from Navigation with Indian Constellations (NavIC) based on GNSS-IR technique. The proposed three steps vegetation correction scheme categorized the crop as per growth stages for sensitivity analysis of NavIC derived multipath phase as GNSS-IR observables and significantly compensated vegetation effects on multipath phase for improved soil moisture retrievals. The proposed method utilized only multipath phase data as single SNR metrics without any ancillary data sets and validated during complete growth cycle of winter wheat crop (sowing to harvesting stages). It has been shown that the soil moisture retrieved using proposed vegetation correction scheme provides a Pearson correlation coefficient of 0.87 and ubRMSD of 0.0341 m3/m3 with in situ measured soil moisture, which is better than bare scheme with Pearson correlation coefficient of 0.71 and ubRMSD of 0.0480 m3/m3. Overall, proposed scheme has found highly effective for vegetation correction and has shown a good potential candidate for VSM retrievals over crop-covered soil using other GNSS constellations (GPS, GLONASS, Galileo and BeiDou etc.).

Unveiling climate change‐induced temperature‐based hotspots across India through multimodel future analysis from CMIP6

AbstractGlobal warming and associated climate change impacts have posed major threats to this 21st century's world, especially for developing countries like India, given various socio‐economic factors. Considering the extensive spatial diversity of this warming‐induced climate changes, suitable region‐specific strategies must be adopted to combat its potential implications. Therefore, it becomes crucial to identify the places most exposed to this changing scenario. This study attempts to shed some light towards this by identifying futuristic “temperature‐based hotspots” across India through a comprehensive multimodel multiscenario analysis at various spatiotemporal scales. A new and more informative index named “Temperature‐based Hotspot Index (THIn)” is proposed for this purpose, which encapsulates various attributes of changing temperature, including its mean, variability and extremes (magnitude, frequency and severity) into a single metric. Bias‐corrected future‐projected temperature data from 14 state‐of‐the‐art general circulation models (GCMs) have been considered for the analysis, which are participating in the coupled model intercomparison project version 6 (CMIP6). The overall analysis identifies temperature‐based hotspots to span predominantly in the west, north and northeast parts of the country. On the other hand, the southern and eastern part of India along the eastern coast are found to be comparatively less exposed to temperature changes in future. Overall, we expect the findings of this study to be beneficial to plan and adopt suitable region‐specific management strategies to combat the challenging future in sufficient advance.

Physical Behavior Profiles Among Older Adults and Their Associations With Physical Capacity and Life-Space Mobility.

We identified data-driven multidimensional physical activity (PA) profiles using several novel accelerometer-derived metrics. Participants aged 75, 80, and 85 (n = 441) wore triaxial accelerometers for 3-7days. PA profiles were formed with k-means cluster analysis based on PA minutes, intensity, fragmentation, sit-to-stand transitions, and gait bouts for men and women. Associations with physical capacity and life-space mobility were examined using age-adjusted general linear models. Three profiles emerged: "Exercisers" and "actives" accumulated relatively high PA minutes, with actives engaging in lighter intensity PA. "Inactives" had the highest activity fragmentation and lowest PA volume, intensity, and gait bouts. Inactives showed lower scores in physical capacity and life-space mobility compared with exercisers and actives. Exercisers and actives had similar physical capacity and life-space mobility, except female exercisers had higher walking speed in the 6-min walk test. Our findings demonstrate the importance of assessing PA as multidimensional behavior rather than focusing on a single metric.

Critical success index or F measure to validate the accuracy of administrative healthcare data identifying epilepsy in deceased adults in Scotland

BackgroundMethods to undertake diagnostic accuracy studies of administrative epilepsy data are challenged by lack of a way to reliably rank case-ascertainment algorithms in order of their accuracy. This is because it is difficult to know how to prioritise positive predictive value (PPV) and sensitivity (Sens). Large numbers of true negative (TN) instances frequently found in epilepsy studies make it difficult to discriminate algorithm accuracy on the basis of negative predictive value (NPV) and specificity (Spec) as these become inflated (usually >90%). This study demonstrates the complementary value of using weather forecasting or machine learning metrics critical success index (CSI) or F measure, respectively, as unitary metrics combining PPV and sensitivity. We reanalyse data published in a diagnostic accuracy study of administrative epilepsy mortality data in Scotland. MethodCSI was calculated as 1/[(1/PPV) + (1/Sens) – 1]. F measure was calculated as 2.PPV.Sens/(PPV + Sens). CSI and F values range from 0 to 1, interpreted as 0 = inaccurate prediction and 1 = perfect accuracy. The published algorithms were reanalysed using these and their accuracy re-ranked according to CSI in order to allow comparison to the original rankings. ResultsCSI scores were conservative (range 0.02–0.826), always less than or equal to the lower of the corresponding PPV (range 39%–100%) and sensitivity (range 2%–93%). F values were less conservative (range 0.039–0.905), sometimes higher than either PPV or sensitivity, but were always higher than CSI. Low CSI and F values occurred when there was a large difference between PPV and sensitivity, e.g. CSI was 0.02 and F was 0.039 in an instance when PPV was 100% and sensitivity was 2%. Algorithms with both high PPV and sensitivity performed best in terms of CSI and F measure, e.g. CSI was 0.826 and F was 0.905 in an instance when PPV was 90% and sensitivity was 91%. ConclusionCSI or F measure can combine PPV and sensitivity values into a convenient single metric that is easier to interpret and rank in terms of diagnostic accuracy than trying to rank diagnostic accuracy according to the two measures themselves. CSI or F prioritise instances where both PPV and sensitivity are high over instances where there are large differences between PPV and sensitivity (even if one of these is very high), allowing diagnostic accuracy thresholds based on combined PPV and sensitivity to be determined. Therefore, CSI or F measures may be helpful complementary metrics to report alongside PPV and sensitivity in diagnostic accuracy studies of administrative epilepsy data.