Set Of Metrics Research Articles

Standardizing Domains and Metrics of Stroke Recovery: A Systematic Review

Background and Aims: Measuring stroke recovery poses a significant challenge, given the complexity of the recovery process. We aimed to identify a standardized and data-driven set of metrics of stroke rehabilitation in the literature that ensures the inclusion of all recovery domains and subdomains in the literature. Methods: A systematic review was conducted by four reviewers using the PRISMA guidelines on PubMed, MEDLINE, and Embase for stroke recovery articles between 2004 and 2024. The inclusion criteria comprised experimental/observational studies, including ischemic and hemorrhagic stroke. All studies had ≥20 participants who were ≥18 years of age, and had a follow-up of ≥3 months. Outcomes included demographics, geographic origin, stroke mechanism, domains and subdomains, metrics used, and follow-up. A bias assessment was performed using the Newcastle-Ottawa Scale and the Cochrane Risk of Bias 2.0 tool. This study was registered with PROSPERO (CRD42024551753). Results: Our search included 324 studies with a sample of 85,156 participants. The study identified seven domains (perception, physical and motor function (PF), speech and language (S&L), cognition, activities of daily living (ADL), quality of life (QoL), and social interaction) and 96 constituent subdomains that encompass the complete landscape of the stroke recovery literature identified. The domains of PF and ADL constituted the vast share of the literature, albeit reducing in their relative representation over time, while domains such as perception and QoL have been increasingly studied since 2004. Using the domains, the study identified the set and frequency of all commonly used metrics to measure stroke recovery in the literature, of which the NIHSS (n = 72), BI (n = 55), and mRS (n = 51) were the most commonly used. We identified eighteen standard metrics that ensure the inclusion of all seven domains and 96 subdomains. Summary of Review and Conclusions: The identified set of domains and metrics within this study can help inform further clinical research and decision-making by providing a standardized set of metrics to be used for each domain. This approach ensures lesser represented domains and subdomains are also included during testing, providing a more complete view and measure of stroke recovery.

Cross entropy and log likelihood ratio cost as performance measures for multi-conclusion categorical outcomes scales.

The inconclusive category in forensics reporting is the appropriate response in many cases, but it poses challenges in estimating an "error rate". We discuss the use of a class of information-theoretic measures related to cross entropy as an alternative set of metrics that allows for performance evaluation of results presented using multi-category reporting scales. This paper shows how this class of performance metrics, and in particular the log likelihood ratio cost, which is already in use with likelihood ratio forensic reporting methods and in machine learning communities, can be readily adapted for use with the widely used multiple category conclusions scales. Bayesian credible intervals on these metrics can be estimated using numerical methods. The application of these metrics to published test results is shown. It is demonstrated, using these test results, that reducing the number of categories used in a proficiency test from five or six to three increases the cross entropy, indicating that the higher number of categories was justified, as it they increased the level of agreement with ground truth.

Lipschitz‐free spaces over strongly countable‐dimensional spaces and approximation properties

AbstractLet be a compact, metrisable and strongly countable‐dimensional topological space. Let be the set of all metrics on compatible with its topology, and equip with the topology of uniform convergence, where the metrics are regarded as functions on . We prove that the set of metrics for which the Lipschitz‐free space has the metric approximation property is residual in .

Re-evaluating the timing of sequential cranial ultrasound screening in very preterm infants for predicting neurodevelopmental outcomes.

Accurate and early prediction of neurodevelopmental impairment is a crucial endeavor in caring for very preterm infants (<31weeks' gestation). Sequential cranial ultrasound is the standard of care for the evaluation of preterm brain injury. However, there is no consensus on the timing and frequency of ultrasound screening. At Izaak Walton Killam (IWK) Health Centre, Halifax, Canada, four-time points for routine ultrasound of very preterm infants are performed at weeks 1, 2, 6, and term age. The hypothesis behind this work is that a three-time-point model will be appropriate for neurodevelopmental impairment prognostication. In this retrospective cohort, all very preterm infants (220-306weeks) born between January 2004 and December 2018 with a neurodevelopmental assessment at 36months corrected age were included. Three prediction models of neurodevelopmental impairment were compared: 1. A reference model including the gestational age, infant sex, and 2-week and 6-week ultrasound 2. A model including the gestational age, infant sex, and 6-week ultrasound 3. A model including the gestational age, infant sex, and 2-week ultrasound RESULTS: Of 786 eligible preterm infants born during the study period, 656/786 survivors were included in the analysis (mean gestational age 275weeks, mean birth weight 1,133g, and 55% male infants). At 36months of corrected age, 30% developed neurodevelopmental impairment. All three models provided comparable discrimination areas under the curve (AUC) of neurodevelopmental impairment at 36months of corrected age. Both the 6-week and the reference model had similar AUC of 0.68 (95% CI 0.63-0.72) and were not noticeably different from the 2-week model (AUC 0.66 (95% CI 0.61-0.70)). The 6-week model provided the best prediction with the lowest Akaike information criterion (AIC) of 766 for the 6-week-only model, AIC 769 for combined weeks 2 and 6 (reference model), and AIC 784 for the 2-week-only model. In this cohort of very preterm infants, a model including 6-week ultrasound only was comparable to a reference model combining 2-week and 6-week ultrasound and showed nearly identical predictive performance of neurodevelopmental impairment at 36months corrected age across a broad set of metrics; thus, it is redundant to do both the 2-week and 6-week ultrasound. Late ultrasound at 6weeks of age provided comparable diagnostic and prognostic information to a reference model combining 2-week and 6-week ultrasound and, if anything, was slightly superior to the 2-week ultrasound model, across a broad set of metrics. The 2-week ultrasound can be eliminated with no impact on the prediction of neurodevelopmental impairment at 36months, promoting prudent resource allocation and stewardship in healthcare.

Computer vision methods for side scan sonar imagery

Abstract Side-scan sonar data presents a significant challenge to analysts due to the size of the material that requires processing. Automatic recognition systems can help in the interpretation and filtration of the data, reducing the time needed for analysis. Deep learning architectures have been employed for computer vision tasks in the underwater domain. The objective of this work is to review current deep learning methods for automatic image classification, object detection, semantic segmentation, and instance segmentation through statistical analysis of published studies. Further, this work includes an inventory of publicly available side-scan sonar data followed by a discussion on augmentation methods applied to side-scan sonar imagery. The study discusses current challenges in the automatic underwater target detection field, highlights main gaps within the domain and identifies potential research directions. Our proposal includes the creation of a standardised benchmark and the use of a common set of metrics allowing for consistent evaluation of future research works.

From Concept to Market: Ensemble Predictive Model for Research Project Crowdfunding Readiness

This study introduces an ensemble model that integrates random forest, gradient boosting, and logistic regression to predict the success of crowdfunding campaigns. Our research develops a novel set of metrics that assess the developmental stage of research projects, facilitating the transition from concept to market-ready product. Utilizing data from multiple sources, including Kaggle’s dataset of Kickstarter and Indiegogo projects and a proprietary dataset tailored to our study, the model’s performance was evaluated against traditional implementations of random forest and gradient boosting. The results demonstrate the ensemble model’s superior performance, achieving an accuracy of 98.94% and an F1 score of 98.81%, significantly outperforming the individual models, showing the best accuracies of around 91% for random forest and lower scores for gradient boosting. This enhancement in predictive power allows for optimized resource allocation and strategic planning in project development, thereby increasing the likelihood of crowdfunding success. This approach streamlines the process of bringing innovative ideas to final products, while at the same time offering a methodologically advanced tool for stakeholders to enhance their campaign strategies effectively.

A comprehensive experience-based framework of actionable priorities to enhance Canada’s health and social care system: discussions with older persons, clinicians, managers, and decision-makers using nominal group technique

BackgroundExisting frameworks of older persons’ experiences in health and social care systems are usually based on single stakeholder perspectives and focused on limited aspects that are independent from one another. This study aimed to describe such experiences from the perspective of diverse stakeholders and develop a comprehensive experience-based framework for actionable priorities to enhance Canada’s health and social care system.MethodsWe adapted a nominal group technique with a qualitative approach. We formed four groups of stakeholders (4 older persons, 7 clinicians, 5 managers, 4 decision-makers) in the province of Quebec. First, participants individually completed a questionnaire to rank the six domains of an international set of metrics of quality of care for older persons in terms of their importance. Then, we conducted a remote discussion with each stakeholder group, obtained consensus on the priority domains, and explored personal/professional experiences explaining prioritization processes and what changes are needed. An inductive-deductive thematic analysis was conducted to develop a framework for actionable health and social care priorities.ResultsThe framework included six key categories revolving around a core category of ‘health equity’. (1) ‘Access to services and care’ focused on improved accessibility of healthcare navigators, primary and home care services, and end-of life care. (2) ‘Respect for older persons’ involved individuality and uniqueness, daily needs, advocacy in defence of vulnerability, and decision-making valuing care preferences. (3) ‘Involvement of family and friends’ encompassed formal recognition, responsibilities, and trajectories associated with caregiving. (4) ‘Communication and education’ consisted of adaptive communication between older persons, caregivers, and professionals and societal-level education and awareness-raising. (5) ‘Coordination and integration of care’ comprised health and social services organization, cross-sectoral approaches, and interdisciplinary collaboration. (6) ‘Responsiveness to evolving needs’ of older persons involved timely response to health transitions and promoting holistic approaches.ConclusionsThis study illustrates the complex challenges of caring for older persons. The framework suggests several areas for improvement in care practices and policies for older persons and caregivers. Proposed intervention targets include public healthcare system navigators, standardized goals of care practices respecting persons’ wishes, formal recognition of caregiver contributions, society-level education, cross-sectoral initiatives, and holistic approaches.

Representation Learning for Sequential Volumetric Design Tasks

Abstract Volumetric design, also called massing design, is the first and critical step in professional building design, which is sequential in nature. As the volumetric design process requires careful design decisions and iterative adjustments, the underlying sequential design process encodes valuable information for designers. Many efforts have been made to automatically generate reasonable volumetric designs, but the quality of the generated design solutions varies, and evaluating a design solution requires either a prohibitively comprehensive set of metrics or expensive human expertise. While previous approaches focused on learning only the final design instead of sequential design tasks, we propose to encode the design knowledge from a collection of expert or high-performing design sequences and extract useful representations using transformer-based models. Later we propose to utilize the learned representations for crucial downstream applications such as design preference evaluation and procedural design generation. We develop the preference model by estimating the density of the learned representations, whereas we train an autoregressive transformer model for sequential design generation. We demonstrate our ideas by leveraging a novel dataset of thousands of sequential volumetric designs. Our preference model can compare two arbitrarily given design sequences and is almost 90% accurate in evaluation against random design sequences. Our autoregressive model is also capable of autocompleting a volumetric design sequence from a partial design sequence.

A Cohesive Deep Drilling Field Strategy for LSST Cosmology

The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) will image billions of astronomical objects in the wide–fast–deep primary survey and in a set of minisurveys including intensive observations of a group of deep drilling fields (DDFs). The DDFs are a critical piece of three key aspects of the LSST Dark Energy Science Collaboration (DESC) cosmological measurements: they provide a required calibration for photometric redshifts (photo-z) and weak gravitational lensing (WL) measurements and they directly contribute to cosmological constraints from the most distant Type Ia supernovae (SNe Ia). We present a set of cohesive DDF strategies fulfilling science requirements relevant to DESC and following the guidelines of the Survey Cadence Optimization Committee. We propose a method to estimate the observing strategy parameters and we perform simulations of the corresponding surveys. We define a set of metrics for each science case to assess the performance of the proposed observing strategies. We show that the most promising results are achieved with deep rolling surveys characterized by two sets of fields: ultradeep fields (z ≲ 1.1) observed at a high cadence with a large number of visits over a limited number of seasons, and deep fields (z ≲ 0.7), observed with a cadence of ∼3 nights for 10 yr. These encouraging results should be confirmed with realistic simulations using the LSST scheduler. A DDF budget of ∼8.5% is required to design observing strategies satisfying all the cosmological requirements. A lower DDF budget leads to surveys that either do not fulfill the photo-z/WL requirements or are not optimal for SN Ia cosmology.

Towards a Common Metrics and Evaluation Framework for Assessment of Older Adults and Caregivers Interacting with Artificial Intelligence

Artificial intelligence (AI) has applications in assisting older adults to age in place and provide support to them and their caregivers as their cognition declines with age. However, effective assessment methods of this technology are needed in order to benchmark their performance and a common set of metrics and evaluation methods would enable such assessments to be compared to one another. To this end, we propose a common framework for human-AI interaction involving care recipients and their care networks. From the results of a literature review exercise, a framework with sample metrics, related measures, qualified evaluation tools, and contextual factors that impact assessment are reviewed. This paper provides a sample of common metrics in one of the framework’s measurement spaces (human-AI interaction) and discusses some of the impacts of contextual factors and how use of the common metrics and evaluation framework can be used for meta-analysis and to guide future research. Additional future articles are planned to cover the other measurement spaces in the framework (system performance, task performance, and well-being), including their particular common metrics and evaluation methods. This effort aims to provide guidance for researchers in this domain as well as highlight measurement gaps that can be filled by future research.

DQM: Data Quality Metrics for AI Components in the Industry

In industrial settings, measuring the quality of data used to represent an intended domain of use and its operating conditions is crucial and challenging. Thus, this paper aims to present a set of metrics addressing this data quality issue in the form of a library, named DQM (Data Quality Metrics), for Machine Learning (ML) use. Additional metrics specific to industrial application are developed in the proposed library. This work aims also to assess various data and datasets types. Those metrics are used to characterize the training and evaluating datasets involved in the process of building ML models for industrial use cases. Two categories of metrics are implemented in DQM: inherent data metrics, are the ones evaluating the quality of a given dataset independently from the ML model such as statistical proprieties and attributes, and model dependent metrics which are those implemented to measure the quality of the dataset by considering the ML model outputs such the gap between two datasets in regards to a given ML model. DQM is used in the scope of the Confiance.ai program to evaluate datasets used for industrial purposes such as autonomous driving.

Quantification of bacterial shape using moment invariants enables distinguishing populations during cellular plasmolysis

The analysis of geometrical cell shape is fundamental to understand motility, development, and responses to external stimuli. The moment invariants framework quantifies cellular shape and size, although its applicability has not been explored for rod-shaped bacteria. In this work, we use moment invariants to evaluate the extent of cell shape change (projected area and volume) during plasmolysis, as Escherichia coli cells are subjected to hyperosmotic shock. The characteristic cell size descriptors width, length and area show systematic decrease as external salt (NaCl) conditions increase—except for high salt, where a small population of cells shows evidence of membrane rupture. We use these two-dimensional results to estimate cell volume during plasmolysis, finding a minimum volume that is not reduced further with increase in salt concentration. Next, we computed elongation and dispersion, metrics that quantify how cell shape is stretched out or differs from an ellipse, respectively. For dispersion, we observe the development of a long tail for the distribution at high salt. Moreover, the use of elongation-dispersion plots enables distinction of plasmolyzed and normal cells despite the presence of broad distributions. Altogether, a protocol is provided to evaluate bacterial shape, highlighting a set of metrics that help distinguish among bacterial populations.•Moment invariants enable quantitative description of bacterial morphology in two dimensions, and estimation of volume•We apply the moment invariants framework to describe changes in bacterial shape during plasmolysis•The proposed methodology shows suitability to distinguish among cellular populations.

Three‐Dimensional Numerical Modeling of Coseismic Atmospheric Dynamics and Ionospheric Responses in Slant Total Electron Content Observations

AbstractDespite routine detection of coseismic acoustic‐gravity waves (AGWs) in Global Navigation Satellite System (GNSS) total electron content (TEC) observations, models of the earthquake‐atmosphere‐ionosphere dynamics, essential for validating data‐driven studies, remain limited. We present the results of three‐dimensional numerical simulations encompassing the entire coupling from Earth's interior to the ionosphere during the 7.8 2016 Kaikoura earthquake. Incorporating the impact of data/model uncertainties in estimating the ionospheric state, the results show a good agreement between observed and simulated slant TEC (sTEC) signals, assessed through a set of metrics. The signals exhibit intricate waveforms, resulting from the integrated nature of TEC and phase cancellation effects, emphasizing the significance of direct signal comparisons along realistic line‐of‐sight paths. By comparing simulation results initialized with kinematic and dynamic source models, the study demonstrates the quantifiable sensitivity of sTEC to AGW source specifications, pointing to their utility in the analysis of coupled dynamics.

FOBICS: Assessing project security level through a metrics framework that evaluates DevSecOps performance

Context:In today’s software development landscape, the DevSecOps approach has gained traction due to its focus on the software development process and bolstering security measures in projects, a task in light of the ever-evolving cybersecurity threats. Objective:This study aims to address the lack of metrics for quantitatively assessing its efficacy from both security and business logic perspectives. Methods:To tackle this issue, the research introduces the Framework of Business Index Concerning Security (FOBICS), a set of metrics designed to enable transparent evaluations of project security. FOBICS considers various perspectives relevant to DevSecOps practices. It includes factors such as project duration and financial outcomes, making it appealing for implementation in business settings. Results:The effectiveness of FOBICS is validated theoretically and empirically via its application in two real-world projects: the results from these implementations show a correlation between FOBICS metrics and the security strategies employed as the development methodologies adopted by diverse teams throughout the projects. Conclusion:Hence, FOBICS emerges as a tool for assessing and continuously monitoring project security, offering insights into areas of strength and areas that may require enhancement. FOBICS is shown to be effective in assessing the level of DevSecOps implementation. The ease of calculating FOBICS metrics makes them easily interpretable and continuously verifiable. Moreover, FOBICS summarizes most of the other quantitative and qualitative metrics in the literature.

Global Metrics for Terrestrial Biodiversity

Biodiversity metrics are increasingly in demand for informing government, business, and civil society decisions. However, it is not always clear to end users how these metrics differ or for what purpose they are best suited. We seek to answer these questions using a database of 573 biodiversity-related metrics, indicators, indices, and layers, which address aspects of genetic diversity, species, and ecosystems. We provide examples of indicators and their uses within the state–pressure–response–benefits framework that is widely used in conservation science. Considering complementarity across this framework, we recommend a small number of metrics considered most pertinent for use in decision-making by governments and businesses. We conclude by highlighting five future directions: increasing the importance of national metrics, ensuring wider uptake of business metrics, agreeing on a minimum set of metrics for government and business use, automating metric calculation through use of technology, and generating sustainable funding for metric production.

Bridging the resolution-sectioning gap in STED nanoscopy with coherent-hybrid depletion.

Microscopes generally achieve better 2D imaging compared to 3D, and super-resolution microscopes often aggravate such axial-lateral resolution mismatch. A prime example is vortex phase-based stimulated emission depletion (STED) fluorescence microscopy, which only narrows the point-spread function laterally, thereby increasing the point-spread function (PSF) anisotropy. In this study, we developed a semi-analytical theory based on the Nijboer-Zernike expansion, developed a set of metrics and performed experiments to establish the merits of the alternative, bivortex phase-based, coherent-hybrid STED. We find that this mode reduces the axial-lateral resolution mismatch by providing access to PSF geometries that are strictly forbidden to the two conventional single-beam modes, emulating noisier multi-beam approaches. Both theory and experiment indicate that bivortex STED not only addresses the axial-lateral resolution gap but also delivers a higher signal-to-background ratio than the two more common STED modes that it bridges.

Sentiment Analysis of Arabic Tweets: Detecting Revilement

Social media systems play an necessary function in shaping public opinion and reflecting societal sentiments. This study focuses on sentiment analysis in Arabic tweets with a particular focus on offensive or offensive content. The aim of this research is to boost a dependable sentiment evaluation model that can accurately classify Arabic tweets as positive or negative, with a particular focus on identifying offensive language. A multinomial Naive Bayes classifier is trained on pre-processed data to perform sentiment classification. The classifier is fine-tuned to differentiate between positive and negative emotions, with a particular focus on identifying offensive or swearing language. The model is evaluated the usage of a complete set of metrics along with precision, precision, recall, and F1score. Experimental consequences point out promising overall performance of the developed sentiment evaluation model. The model achieved an accuracy of 93%, effectively classifying Arabic tweets into effective and bad sentiments. The precision, recall, and F1-score metrics similarly validate the model's capacity to precisely become aware of revilement and offensive language. These outcomes spotlight the conceivable of the proposed strategy in successfully examining Arabic tweets for sentiment and offensive content, contributing to higher grasp on line behaviors and sentiments in the context of revilement.

On the use of Change History Data to Enhance Class Change-Proneness Prediction Models

As software evolves, new artifacts are created, modified, or removed. One of these main artifacts generated in the development of object-oriented software is the class. Classes have a very dynamic life cycle that can result in additional costs to the project. One way to mitigate this is to detect, in the early stages of the development, classes that are prone to change. Some approaches in the literature adopt Machine Learning (ML) algorithms to predict the change-proneness of a class. However, most of these approaches do not consider the temporal dependency between training instances, i.e., they consider that the instances are independent. To overcome such a limitation, this study presents an approach for predicting change-proneness based on the class change history. The approach adopts the sliding window method and is evaluated to obtain six kinds of models, which are derived by using, as predictors, different sets of metrics: structural, evolutionary, and smell-based. The evaluation uses five systems, four ML algorithms, and also explores some resample techniques to deal with imbalanced data. Regardless of the kind of model analyzed and the algorithm used, our approach overcomes the traditional one in 378 (~80) cases, out of 420, considering all systems, kinds of models, indicators, and algorithms. Moreover, the results show that our approach presents the best performance when the set of evolutionary metrics is used as predictors. There is no improvement when smell-based metrics are added. The Random Forest algorithm with the resampling technique ADA reaches the best performance among the ML algorithms evaluated.

On the significance of the different geometrical and dosimetric parameters in microbeam and minibeam radiation therapy a retrospective evaluation.

Spatially Fractionated Radiation Therapy (SFRT) is an unconventional therapeutic approach with the potential to disrupt the classical paradigms of conventional radiation therapy. The high spatial dose modulation in SFRT is believed to activate distinct radiobiological mechanisms which lead to remarkable increases in normal tissue tolerance. To make optimal use of SFRT and its benefits, a deeper understanding of the biological response and its relationship with the complex dosimetric and geometric components of SFRT is essential. A retrospective evaluation of preclinical studies was conducted to gain insight into the dosimetric and geometric parameters that are most correlated with normal tissue response. Current literature evaluates the response of tissue to MBRT and MRT according to various end points, e.g. the level of desquamation, degree of necrosis, or the amount of malcalcification. A set of metrics was developed to allow a quantitative comparison of these results. The strongest correlations were observed with the doses in both the peaks and valleys as well as the ratio of the area covered by the peak over the total area. This emphasises the geometry of the beam. MBRT challenged previous uniform dose-distribution paradigms by highlighting the critical role of Peak Dose alongside Valley Dose in tissue sparing whereas MRT underscores the significant influence of geometric beam parameters on tissue preservation. The data exhibits variability in the results obtained using different animal models and endpoints and additional research is warranted to explore the trends observed in this study under controlled conditions.

Empowering sign language communication: Integrating sentiment and semantics for facial expression synthesis

Translating written sentences from oral languages to a sequence of manual and non-manual gestures plays a crucial role in building a more inclusive society for deaf and hard-of-hearing people. Facial expressions (non-manual), in particular, are responsible for encoding the grammar of the sentence to be spoken, applying punctuation, pronouns, or emphasizing signs. These non-manual gestures are closely related to the semantics of the sentence being spoken and also to the utterance of the speaker’s emotions. However, most Sign Language Production (SLP) approaches are centered on synthesizing manual gestures and do not focus on modeling the speaker’s expression. This paper introduces a new method focused in synthesizing facial expressions for sign language. Our goal is to improve sign language production by integrating sentiment information in facial expression generation. The approach leverages a sentence’s sentiment and semantic features to sample from a meaningful representation space, integrating the bias of the non-manual components into the sign language production process. To evaluate our method, we extend the Fréchet gesture distance (FGD) and propose a new metric called Fréchet Expression Distance (FED) and apply an extensive set of metrics to assess the quality of specific regions of the face. The experimental results showed that our method achieved state of the art, being superior to the competitors on How2Sign and PHOENIX14T datasets. Moreover, our architecture is based on a carefully designed graph pyramid that makes it simpler, easier to train, and capable of leveraging emotions to produce facial expressions. Our code and pretrained models will be available at: https://github.com/verlab/empowering-sign-language.