Concrete Metrics Research Articles

Natural regeneration of drylands and associated pathways to human health outcomes: Perspectives from rural households

Land degradation and climate change are intertwined global challenges, their implications on human health are driven by the unsustainable use of natural resources, resulting in the loss of vital ecosystem services that support human health. There are equity implications in that landscape degradation disproportionately affects vulnerable populations that live in the most fragile ecosystems. Solutions to reverse degradation are increasingly considered attractive because of their ability to meet multiple societal objectives, however, their implementation has outpaced research on impact; the research and evidence gaps are particularly acute in relation to quantifying the health and well-being impacts of nature-based solutions. This study documents the experiences of rural communities implementing Farmer Managed Natural Regeneration (FMNR) in Central Tanzanian drylands, which integrates trees in agricultural landscapes. FMNR technique has been hailed as the largest positive environmental transformation in African drylands due to its effectiveness in restoring landscapes at very low costs. Despite its success, the impact on human well-being is largely unquantified. We used in-depth group discussions across four villages to document how respondents perceive the health and well-being impacts of natural regeneration. We thereafter adapted the ecosystem services cascade model to conceptualise the pathways between natural regeneration and impact on health and well-being. Respondents across all study sites unanimously reported how FMNR implementation has reversed decades of land degradation, resulting in physical and mental health benefits that can be linked to food and nutritional security, improved air and water quality, income diversification, heat adaptation and gender equality. We demonstrate how the application of the cascade model can generate causal pathways that 1) map how changes in ecosystem structure, functions and services can result in measurable health outcomes, and, 2) support empirical investigation by defining concrete metrics for monitoring and evaluation of interventions.

Quantifying the ultrafast and steady-state molecular reduction potential of a plasmonic photocatalyst.

Plasmonic materials are promising photocatalysts as they are well suited to convert light into hot carriers and heat. Hot electron transfer is suggested as the driving force in many plasmon-driven reactions. However, to date, there are no direct molecular measures of the rate and yield of plasmon-to-molecule electron transfer or energy of these electrons on the timescale of plasmon decay. Here, we use ultrafast and spectroelectrochemical surface-enhanced Raman spectroscopy to quantify electron transfer from a plasmonic substrate to adsorbed methyl viologen molecules. We observe a reduction yield of 2.4 to 3.5% on the picosecond timescale, with plasmon-induced potentials ranging from [Formula: see text]3.1 to [Formula: see text]4.5 mV. Excitingly, some of these reduced species are stabilized and persist for tens of minutes. This work provides concrete metrics toward optimizing material-molecule interactions for efficient plasmon-driven photocatalysis.

Framework for Guiding the Development of High-Quality Conversational Agents in Healthcare.

Evaluating conversational agents (CAs) that are supposed to be applied in healthcare settings and ensuring their quality is essential to avoid patient harm and ensure efficacy of the CA-delivered intervention. However, a guideline for a standardized quality assessment of health CAs is still missing. The objective of this work is to describe a framework that provides guidance for development and evaluation of health CAs. In previous work, consensus on categories for evaluating health CAs has been found. In this work, we identify concrete metrics, heuristics, and checklists for these evaluation categories to form a framework. We focus on a specific type of health CA, namely rule-based systems that are based on written input and output, have a simple personality without any kind of embodiment. First, we identified relevant metrics, heuristics, and checklists to be linked to the evaluation categories through a literature search. Second, five experts judged the metrics regarding their relevance to be considered within evaluation and development of health CAs. The final framework considers nine aspects from a general perspective, five aspects from a response understanding perspective, one aspect from a response generation perspective, and three aspects from an aesthetics perspective. Existing tools and heuristics specifically designed for evaluating CAs were linked to these evaluation aspects (e.g., Bot usability scale, design heuristics for CAs); tools related to mHealth evaluation were adapted when necessary (e.g., aspects from the ISO technical specification for mHealth Apps). The resulting framework comprises aspects to be considered not only as part of a system evaluation, but already during the development. In particular, aspects related to accessibility or security have to be addressed in the design phase (e.g., which input and output options are provided to ensure accessibility?) and have to be verified after the implementation phase. As a next step, transfer of the framework to other types of health CAs has to be studied. The framework has to be validated by applying it during health CA design and development.

What Can We Learn from Quality Requirements in ISO/TS 82304-2 for Evaluating Conversational Agents in Healthcare?

Evaluating conversational agents (CA) that are supposed to be applied in healthcare and ensuring their quality is essential to avoid patient harm. However, most researchers only study usability and use the CA in clinical trials before conducting such careful evaluation. In previous work, consensus on metrics for evaluating healthcare CA have been found. However, the metrics are still too generic to form an evaluation framework. In this work, we try to link the ISO technical specification ISO/TS 82304-2 Quality Requirements for Health and Wellness Apps to the set of metrics to come a step closer towards an evaluation framework. We identify three links between ISO requirements and the set of metrics, namely accessibility, usability, and security. Although the technical specification rather lists aspects to be considered during development instead of concrete metrics for studying the quality, we can link to some aspects that are also of interest for health CA evaluation. For example, measuring the readability for ensuring accessibility or implementing the Web Content Accessibility Guidelines are two aspects of relevance for health CA.

Benchmarking for Observability: The Case of Diagnosing Storage Failures

Diagnosing storage system failures is challenging even for professionals. One recent example is the “When Solid State Drives Are Not That Solid” incident occurred at Algolia data center, where Samsung SSDs were mistakenly blamed for failures caused by a Linux kernel bug. With the system complexity keeps increasing, diagnosing failures will likely become more difficult.To better understand real-world failures and the potential limitations of state-of-the-art tools, we first conduct an empirical study on 277 user-reported storage failures in this paper. We characterize the issues along multiple dimensions (e.g., time to resolve, kernel components involved), which provides a quantitative measurement of the challenge in practice. Moreover, we analyze a set of the storage issues in depth and derive a benchmark suite called BugBenchk. The benchmark suite includes the necessary workloads and software environments to reproduce 9 storage failures, covers 4 different file systems and the block I/O layer of the storage stack, and enables realistic evaluation of diverse kernel-level tools for debugging.To demonstrate the usage, we apply BugBenchk to study two representative tools for debugging. We focus on measuring the observations that the tools enable developers to make (i.e., observability), and derive concrete metrics to measure the observability qualitatively and quantitatively. Our measurement demonstrates the different design tradeoffs in terms of debugging information and overhead. More importantly, we observe that both tools may behave abnormally when applied to diagnose a few tricky cases. Also, we find that neither tool can provide low-level information on how the persistent storage states are changed, which is essential for understanding storage failures. To address the limitation, we develop lightweight extensions to enable such functionality in both tools. We hope that BugBenchk and the enabled measurements will inspire follow-up research in benchmarking and tool support and help address the challenge of failure diagnosis in general.

Adaptation of Kirkpatrick’s Four-Level Model of Training Criteria to Evaluate Training Programmes for Head Teachers

Training programmes are evaluated to verify their effectiveness, assess their ability to achieve their goals and identify the areas that require improvement. Therefore, the target of evaluators is to develop an appropriate framework for evaluating training programmes. This study adapted Kirkpatrick’s four-level model of training criteria published in 1959 to evaluate training programmes for head teachers according to their own perceptions and those of their supervisors. The adapted model may help evaluators to conceptualise the assessment of learning outcomes of training programmes with metrics and instruments. The model also helps to determine the strengths and weaknesses of the training process. The adaptation includes concrete metrics and instruments for each of the four levels in the model: reaction criteria, learning criteria, behaviour criteria and results criteria. The adapted model was applied to evaluate 12 training programmes for female head teachers in Saudi Arabia. The study sample comprised 250 trainee head teachers and 12 supervisors. The results indicated that the adapted Kirkpatrick evaluation model was very effective in evaluating educational training for head teachers.

EUKulele: Taxonomic annotation of the unsung eukaryotic microbes

The assessment of microbial species biodiversity is essential in ecology and evolutionary biology (Reaka-Kudla et al. 1996), but especially challenging for communities of microorganisms found in the environment (Das et al. 2006, Hillebrand et al. 2018). Beyond providing a census of organisms in the ocean, assessing marine microbial biodiversity can reveal how microbes respond to environmental change (Salazar et al. 2017), clarify the ecological roles of community members (Hehemann et al. 2016), and lead to biotechnology discoveries (Das et al. 2006). Computational approaches to characterize taxonomic diversity and phylogeny based on the quality of available data for environmental sequence datasets is fundamental for advancing our understanding of the role of these organisms in the environment. Even more pressing is the need for comprehensive and consistent methods to assign taxonomy to environmentally-relevant microbial eukaryotes. Here, we present EUKulele, an open-source software tool designed to assign taxonomy to microeukaryotes detected in meta-omic samples, and complement analysis approaches in other domains by accommodating assembly output and providing concrete metrics reporting the taxonomic completeness of each sample.

Implementation of the Institute of Medicine treatment plan in oncology practice.

e19233 Background: Participation in the Center for Medicare and Medicaid Services (CMS) value-based payment reform, The Oncology Care Model, requires that every beneficiary has a documented 13-point Institute of Medicine (IOM) treatment plan (TP) when commencing antineoplastic therapy. The intent of this document is to enhance shared decision making between the patient and care team by providing transparent treatment recommendations and engaging patients and caregivers in meaningful discussion. There is limited discussion in the literature on how to adapt the CMS recommendations to diverse practice settings while maintaining fidelity to the intent of the TP. Methods: We compare how three clinically and geographically unique OCM participating institutions implemented the TP in their respective institutions within the domains of the Consolidated Framework for Implementation Research (CFIR). Settings include a community cancer institute in the northeast, an academic hospital setting in the southeast, and a large community cancer network in the southern United States. Results: We identified similar themes in implementation including engaging stakeholders, leveraging information technology, structuring the TP, development of clinic processes and considering scalability. We also describe adaptations unique to the culture and setting of each site. Conclusions: Although studies have shown patients do not feel informed of their diagnosis, there are currently many approaches to improving shared decision making including utilizing the 13 points of the IOM TP as mandated by the OCM. We provide practical strategies for incorporation of the TP into clinical care with lessons from diverse settings. As shown by the wide variability in implementing shared decision making, further research is needed to optimize illness understanding. Additionally, optimal implementation of CMS’s IOM TP would ideally include concrete metrics measuring impact on shared decision making, illness understanding, or patient satisfaction.

Helping the Reality of Materials Sustainability Bloom

How we can break through the noise of vague and untrustworthy climate change messaging with specific and provable sustainability metrics?

Introducing risk inequality metrics in tuberculosis policy development

Global stakeholders including the World Health Organization rely on predictive models for developing strategies and setting targets for tuberculosis care and control programs. Failure to account for variation in individual risk leads to substantial biases that impair data interpretation and policy decisions. Anticipated impediments to estimating heterogeneity for each parameter are discouraging despite considerable technical progress in recent years. Here we identify acquisition of infection as the single process where heterogeneity most fundamentally impacts model outputs, due to selection imposed by dynamic forces of infection. We introduce concrete metrics of risk inequality, demonstrate their utility in mathematical models, and pack the information into a risk inequality coefficient (RIC) which can be calculated and reported by national tuberculosis programs for use in policy development and modeling.

Quantifying the effect of construction site factors on concrete compressive strength using designed experiments

Both structured and unstructured factors affect concrete compressive strength. Structured factors, e.g., raw materials, affect concrete production. Unstructured factors, e.g., local conditions, affect concrete quality during the construction phase. The effects of structured factors on concrete metrics are well understood, while there is limited understanding on the effects of unstructured factors. A full 25 factorial design was conducted to quantify the effect of five construction site factors including crew experience, compaction method, mixing time, curing humidity and curing temperature. A linear regression model was developed considering significant affecting factors. Model adequacy was evaluated through residual plots. The results indicate that compaction method, mixing time, curing humidity and curing temperature affect concrete compressive strength significantly, with curing temperature having the highest percent contribution (50.0%). The final regression model was used to create a decision-support tool that enables construction workers to find operating conditions and take corrective actions to preserve concrete quality.

Circular economy as an essentially contested concept

The Circular Economy (CE) is currently a popular notion within the policy and business advocacy groups. Despite being visionary and provocative in its message, the research on the CE concept is emerging. The two intertwined objectives of the paper are; first to identify, discuss and develop the various definitions provided by the emerging literature. Secondly, to suggest an initial research approach with which research on CE can be conducted. Our analysis shows that the existing CE work is mainly done on the practical and technical levels of the actual physical flows of materials and energy in production-consumption systems. The focus of the extant literature is on concrete metrics, tools, instruments and computations. Therefore, the basic assumptions concerning the values, societal structures, cultures, underlying world-views and the paradigmatic potential of CE remain largely unexplored. We argue that CE has already become what Gallie (1955) more than six decades ago termed as an “essentially contested concept” (ECC). The paper further suggests a model for CE research that helps in the categorization, classification and organization of research and investigation on CE. The model can help in limiting the observed unbalance and enhance the contribution of the CE approach to a more sustainable global society.

Management of Service Level Agreements for Cloud Services in IoT: A Systematic Mapping Study

Cloud computing and Internet of Things (IoT) are computing technologies that provide services to consumers and businesses, allowing organizations to become more agile and flexible. Therefore, ensuring quality of service (QoS) through service-level agreements (SLAs) for such cloud-based services is crucial for both the service providers and service consumers. As SLAs are critical for cloud deployments and wider adoption of cloud services, the management of SLAs in cloud and IoT has thus become an important and essential aspect. This paper investigates the existing research on the management of SLAs in IoT applications that are based on cloud services. For this purpose, a systematic mapping study (a well-defined method) is conducted to identify the published research results that are relevant to SLAs. This paper identifies 328 primary studies and categorizes them into seven main technical classifications: SLA management, SLA definition, SLA modeling, SLA negotiation, SLA monitoring, SLA violation and trustworthiness, and SLA evolution. This paper also summarizes the research types, research contributions, and demographic information in these studies. The evaluation of the results shows that most of the approaches for managing SLAs are applied in academic or controlled experiments with limited industrial settings rather than in real industrial environments. Many studies focus on proposal models and methods to manage SLAs, and there is a lack of focus on the evolution perspective and a lack of adequate tool support to facilitate practitioners in their SLA management activities. Moreover, the scarce number of studies focusing on concrete metrics for qualitative or quantitative assessment of QoS in SLAs urges the need for in-depth research on metrics definition and measurements for SLAs.

On tackling quality threats for the assessment of measurement programs: A case study on the distribution of metric usage and knowledge

Context: Measurement programs are subject to changing requirements, causing the need for a robustness to compensate these changes. Existing methods enable the assessment of measurement programs and their robustness with the help of company representatives. Goal: However, it is not clear how much the spread of knowledge among roles and differences between company internal organizations influence the result and quality of the assessment. Thus, it is not known today who and how many representatives should be interviewed to get reliable results. Method: To address these points we performed a case study on the assessment method MeSRAM, spanning 18 interviews with 14 representatives of 4 different organizations within a large international communication company. Results: Managers are able to answer more questions and their answers are more often correct than those of engineers. Furthermore, most errors occur for questions about the use of concrete metrics, while for aspects about the metrics' infrastructure no conflicting answers occurred. We also identified 6 reasons for gaining incorrect answers. Conclusion: The assessment of measurement programs requires additional care to ensure the quality of the results. • A case study on the assessment of software measurement programs. • Investigation of quality threats to that arise due to knowledge distribution. • A first list of reasons for assessment mistakes and lessons learned.

The Future of the Forensic Pathology Workforce

According to many published reports, testimonials, research studies, and data collected, there are not enough forensic pathologists to serve the United States. Forensic pathologists are responsible for the medicolegal investigation of death. Such investigation includes death scene, history, autopsy, and determining the cause and manner of death. To get an idea of the workload of the forensic pathologist, retrospective data can be examined. The numbers alone reveal that there has been and remains a shortage in the forensic pathologist workforce. If the overall goal is to remedy this forensic pathologist shortage, strategies must be broad and long-term. Also, not all goals will have concrete metrics that can be quantified, such as to increase and improve public image. Some of the underlying problems, trends, future predictions, and possible solutions are discussed.

Towards Systematic Reuse of Metric Specifications

This paper presents a novel approach for reusing metrics. Contrasting a lot of work related to this issue, we are focusing on reusing metric specifications. By introducing reusability concepts such as genericity and variation points to metric specification we enable the creation of reusable ones. On the other hand this allows deriving project specific concrete metrics based on reusable metric specifications. Early experience from our industry cooperation is promising and indicates less stress and effort for the metric users.

The Case for a Contemporary Kazantzakis

The Case for a Contemporary Kazantzakis Franklin Hess Peter Bien , editor and translator. The Selected Letters of Nikos Kazantzakis. Princeton: Princeton University Press, 2012. Pp. 875. Hardcover and eBook. $99.50. Thanasis Maskaleris , translator and editor. The Terrestrial Gospel of Nikos Kazantzakis: Will Humans Be the Saviors of the Earth. Ithaca, NY: Zorba Press, 2011. Pp. 162. Paperback. $20. EBook. $7.99. In recent years, the place of literature in the curriculum and in academic publishing has been steadily eroding. This has been true of the academy as a whole; it has also been true for Modern Greek Studies. From the mid-1960s through the 1980s, literature was the growth sector for our field. Today, new tenure-track positions are concentrated in history and other social sciences. Against this backdrop, we have the publication of two books on Nikos Kazantzakis by scholars who came of age during the heyday of literature in Modern Greek Studies: The Selected Letters of Nikos Kazantzakis (Princeton, 2011), translated and edited by Peter Bien, and The Terrestrial Gospel of Nikos Kazantzakis: Will Humans Be the Saviors of the Earth, translated and edited by Thanasis Maskaleris. Their appearance at this juncture in time begs the question: are these books a requiem for a passing era or, perhaps, the herald of a new one? The reasons for this shift away from literature are multiple and complex: they are generational, institutional, and cultural. Though a full discussion of the crisis is beyond the scope of this review and would also be largely redundant given the publication of a special section on this topic in the May 2011 Journal of Modern Greek Studies, I want to highlight, for the purposes of this essay, a perspective that I believe has been largely neglected in this discussion: that of the student. For starters, there has been a general, culture-wide turn away from print-based media to visual, acoustic, and electronic media.1 Our students reflect this transition. They come to college literate in Buffy, the Vampire Slayer and Twilight; Mad Men; and rap music. They are largely ignorant, however, about Wollstonecraft Shelley, Shakespeare, and Ellison. Not surprisingly, Film and Media Studies have been flourishing within this environment, and Comparative Literature Studies has been contracting. Additionally, the rise of critical theory in literary studies has not been without consequences for our students.2 Theory has provided many important critical insights, vigorous debates, and ample opportunities to publish, something that was of crucial importance during an era where tenure and retention decisions [End Page 339] were being made increasingly in terms of concrete metrics. Theory, however, has also presented challenges to classroom instruction. At its best, it stimulated conversation and provided new vantage points for thinking about traditional texts. At its worst, it created layers of empty abstraction and emphasized micro-political battles that distanced and even alienated large numbers of students from the literary texts they encountered. Perhaps the most important of the reasons for the shift away from literature, however, is the one that Gregory Jusdanis identifies in Fiction Agonistes: In Defense of Literature: the failure to provide a compelling, contemporary justification for the study of literature. While we have deconstructed inherent justifications of literature, we have neither been willing nor able to offer others in their place at any level of education, from elementary school to university. (2010:2) We have failed, in other words, to justify the study of literature not only to administrators and other decision makers, but to students as well. Literary humanism has been roundly criticized and rejected. It frequently remains, however, the de facto justification for the profession. It certainly hasn't been replaced in the minds of our students. They still come to the classroom hoping for enlightenment through encounters with superior minds. Too often they leave disappointed and jaded. Jusdanis proposes a corrective for this state of affairs: that we locate the justification for literature in its parabatic potential, the sharpening of analytic skills that occurs as we move from the fictional realm of the text to the concrete social realm of its production and consumption (2010:3). The movement between literature as autonomous art and social contexts in which it is...

International Health Regulations—What Gets Measured Gets Done

The global spread of severe acute respiratory syndrome highlighted the need to detect and control disease outbreaks at their source, as envisioned by the 2005 revised International Health Regulations (IHR). June 2012 marked the initial deadline by which all 194 World Health Organization (WHO) member states agreed to have IHR core capacities fully implemented for limiting the spread of public health emergencies of international concern. Many countries fell short of these implementation goals and requested a 2-year extension. The degree to which achieving IHR compliance will result in global health security is not clear, but what is clear is that progress against the threat of epidemic disease requires a focused approach that can be monitored and measured efficiently. We developed concrete goals and metrics for 4 of the 8 core capacities with other US government partners in consultation with WHO and national collaborators worldwide. The intent is to offer an example of an approach to implementing and monitoring IHR for consideration or adaptation by countries that complements other frameworks and goals of IHR. Without concrete metrics, IHR may waste its considerable promise as an instrument for global health security against public health emergencies.

Quantification and statistical significance analysis of group separation in NMR-based metabonomics studies

Currently, no standard metrics are used to quantify cluster separation in PCA or PLS-DA scores plots for metabonomics studies or to determine if cluster separation is statistically significant. Lack of such measures makes it virtually impossible to compare independent or inter-laboratory studies and can lead to confusion in the metabonomics literature when authors putatively identify metabolites distinguishing classes of samples based on visual and qualitative inspection of scores plots that exhibit marginal separation. While previous papers have addressed quantification of cluster separation in PCA scores plots, none have advocated routine use of a quantitative measure of separation that is supported by a standard and rigorous assessment of whether or not the cluster separation is statistically significant. Here quantification and statistical significance of separation of group centroids in PCA and PLS-DA scores plots are considered. The Mahalanobis distance is used to quantify the distance between group centroids, and the two-sample Hotelling's T 2 test is computed for the data, related to an F-statistic, and then an F-test is applied to determine if the cluster separation is statistically significant. We demonstrate the value of this approach using four datasets containing various degrees of separation, ranging from groups that had no apparent visual cluster separation to groups that had no visual cluster overlap. Widespread adoption of such concrete metrics to quantify and evaluate the statistical significance of PCA and PLS-DA cluster separation would help standardize reporting of metabonomics data.

Predicting the Success of an Anesthesiology Trainee

Predicting the Success of an Anesthesiology Trainee