Successful Metrics Research Articles

An evaluation tool for assessing coral restoration efforts

The ever-increasing need for coral restoration as a tool available to mitigate reef declines and aid in the recovery of lost ecosystem services requires improving restoration performance over time through an adaptive management framework to evaluate the status of restoration programs using uniform, consistent metrics. An evaluation tool, presented herein, allows restoration practitioners and managers to self-evaluate the robustness of each project and identify successful metrics, those metrics that need special attention, and changes to restoration strategies that can improve performance and aid recovery. This tool is designed to allow programs to track the progress of each key metric over time to assist in improving upon successes and learning from failures. The metrics within this restoration evaluation tool focus on published best-management practices and have resulted from extensive research conducted by restoration experts over the past 20 years. Common metrics of growth and survival are included, in addition to parameters vital to the operational success of restoration programs, such as coral reproduction, recruitment of associated reef taxa, increasing habitat for reef fisheries, and improving overall reef habitat. Five Caribbean restoration programs, each with at least 15 years of restoration experience, are presented as case studies. Each program was evaluated based on six restoration categories including: field-based nurseries, outplantings, programmatic management, education and outreach, event-driven restoration, and socioeconomic restoration. Category-specific metrics were scored with a binary scoring system and summarized using a stop-light indicator framework, where the resulting color/score indicates the operations tatus of the different program components (Scores >75% = green/successful; 50–74.9% = light green > yellow > orange/intermediate; <49.9% = red/sub-optimal). Composite scores may be used to evaluate individual projects, overall restoration programs, or even large-scale state of regional restoration plans. Overall, four of the five programs scored >75% indicating most of these programs are performing well, are versatile, well managed, and sustainable. Outside of environmental factors and large-scale disturbance events, many programs described resource limitations, including funding and staffing, as reasons for scoring low on some metrics. A holistic evaluation rubric incorporated into programmatic self-assessment can ensure that restoration projects and programs are working towards success and sustainability.

The influence of the number and distribution of background points in presence-background species distribution models

Species distribution models (SDMs), which relate recorded observations (presences) and absences or background points to environmental characteristics, are powerful tools used to generate hypotheses about the biogeography, ecology, and conservation of species. Although many researchers have examined the effects of presence and background point distributions on model outputs, they have not systematically evaluated the effects of various methods of background point sampling on the performance of a single model algorithm across many species. Therefore, a consensus on the preferred methods of background point sampling is lacking. Here, we conducted presence-background SDMs for 20 vertebrate species in North America under a variety of background point conditions, varying the number of background points used, the size of the buffer used to constrain the background points around the occurrences, and the percentage of background points sampled within the buffer (“spatial weighting”). We evaluated the accuracy and transferability of the models using Boyce index, overlap with expert-generated range maps, and area overpredicted and underpredicted by the SDM (and AUC for comparability with other studies).SDM performance is highly dependent on the species modelled but is affected by the number and spread of background points. Models with little spatial weighting had high accuracy (overlap values), but extreme extrapolation errors and overprediction. In contrast, SDMs with high transferability (high Boyce index values and low overprediction) had moderate-to-high spatial weighting. These results emphasize the importance of both background points and evaluation metric selection in SDMs. For other, more successful metrics, using many background points with spatial weighting may be preferred for models with large extents. These results can assist researchers in selecting the background point parameters most relevant for their research question, allowing them to fine-tune their hypotheses on the distribution of species through space and time.

Gardening Pocillopora spp. fragments and their potential for rebuilding reef systems in the southern Mexican Pacific

The degradation of coral reefs during the last decades has turned attention toward management and restoration interventions. This study seeks to operationalize coral gardening of Pocillopora spp. fragments in low profile bottom‐anchored nurseries and to compare survival and growth patterns between sites, time, and fragment size in the southern Mexican Pacific. After 357 days, fragments showed high survival (96.4%), growth rates (4.35 cm/year), and an increase in coral coverage from 3.62 ± 1.3% (mean ± SD) to 17.42 ± 4.8% (approximately 480%). Coral growth rate did not show differences between sites (pseudo‐F[1,635] = 0.21, p = 0.63), but corals grew more significantly during the upwelling season in the Gulf of Tehuantepec. According to linear and local regression analysis, the extension rate in nurseries was significantly higher when corals were smaller (<7 cm in diameter) being of relevance for operationalizing coral gardening from donor colonies; nevertheless, coral shrinkage (13.1%), when corals were smaller (i.e. during the early phases of the coral gardening protocol), calls for precaution and close monitoring. Operationalizing coral gardening of Pocillopora spp. fragments, including successful metrics in upwelling areas, are relevant for reef restoration in the eastern tropical Pacific; nevertheless, lessons regarding shrinkage and differential growth rates related to coral size should also be considered.

Validation analysis of EMDB entries.

The Electron Microscopy Data Bank (EMDB) is the central archive of the electron cryo-microscopy (cryo-EM) community for storing and disseminating volume maps and tomograms. With input from the community, EMDB has developed new resources for the validation of cryo-EM structures, focusing on the quality of the volume data alone and that of the fit of any models, themselves archived in the Protein Data Bank (PDB), to the volume data. Based on recommendations from community experts, the validation resources are developed in a three-tiered system. Tier 1 covers an extensive and evolving set of validation metrics, including tried and tested metrics as well as more experimental ones, which are calculated for all EMDB entries and presented in the Validation Analysis (VA) web resource. This system is particularly useful for cryo-EM experts, both to validate individual structures and to assess the utility of new validation metrics. Tier 2 comprises a subset of the validation metrics covered by the VA resource that have been subjected to extensive testing and are considered to be useful for specialists as well as nonspecialists. These metrics are presented on the entry-specific web pages for the entire archive on the EMDB website. As more experience is gained with the metrics included in the VA resource, it is expected that consensus will emerge in the community regarding a subset that is suitable for inclusion in the tier 2 system. Tier 3, finally, consists of the validation reports and servers that are produced by the Worldwide Protein Data Bank (wwPDB) Consortium. Successful metrics from tier 2 will be proposed for inclusion in the wwPDB validation pipeline and reports. The details of the new resource are described, with an emphasis on the tier 1 system. The output of all three tiers is publicly available, either through the EMDB website (tiers 1 and 2) or through the wwPDB ftp sites (tier 3), although the content of all three will evolve over time (fastest for tier 1 and slowest for tier 3). It is our hope that these validation resources will help the cryo-EM community to obtain a better understanding of the quality and of the best ways to assess the quality of cryo-EM structures in EMDB and PDB.

Lean into Clinical Pharmacy: An Experience in Implementing Key Performance Indicators and Gemba Walks into Clinical Pharmacy Services.

Purpose: The Lean methodology was applied to clinical metrics by a critical care pharmacy team. The experiences associated with the development and implementation of clinical metrics and their impact on daily workflow are described. Summary: The Lean methodology has been introduced into the healthcare system as a means of process improvement, which can eliminate waste through appropriate medication utilization. At OhioHealth Riverside Methodist Hospital, the department of pharmacy was tasked with the development of clinical metrics after a health system wide Gemba walk was initiated. The pharmacy department's critical care team developed a strategy identifying and evaluating clinical metrics pertaining to their everyday workflow. Each clinical metric was evaluated in accordance with a pre-defined goal. Metrics requiring heavy documentation and those in which the pharmacist does not have autonomous authority to manage were often challenging to implement and were less successful. Throughout this process, the lessons learned focused on generating ideas that were easily documented, evidence-based, and department specific. The critical care team discovered that the outcome of the most successful metrics highlighted clinical pharmacist value and data generated could be used to support funding for additional resources. Conclusion: The critical care pharmacy team developed a streamlined process to implement clinical metrics as means of identifying areas for improvement using the Lean methodology.

MeTeaM—A method for characterizing mature software metrics teams

Background:Metrics teams play an increasingly important role in handling data and information in modern software development organizations; they manage their companies’ measurement programs, collect and process data, and develop and distribute information products. Metrics teams can comprise several roles, and their set-up can differ between companies, as can the metrics maturity of host organizations. These differences impact the effectiveness and quality of a team’s measurement program. Objective:Our objective was to design and evaluate a model to describe the characteristics of a mature metrics team, which efficiently designs, develops, maintains, and evolves its organization’s measurement program. Method:We conducted an action research study on four metrics teams of four distinct companies. We designed and evaluated a domain-specific model for assessing the maturity of metrics teams – MeTeaM – and also assessed the four metrics teams per se. Results:Our results were two-fold: the creation of the metrics team maturity model MeTeaM and a template to assess metrics teams. Our evaluation showed that the model captures the characteristics of successful metrics teams and quantifies the maturity status of both the metrics teams and their host organizations. Conclusions:More mature metrics teams score higher in the MeTeaM model than less mature teams. The assessment provides less mature metrics teams with valuable insights on what factors to improve. Such insights can be shared with and acted upon successfully with their organizations.

New Developer Metrics for Open Source Software Development Challenges: An Empirical Study of Project Recommendation Systems

Software collaboration platforms where millions of developers from diverse locations can contribute to the common open source projects have recently become popular. On these platforms, various information is obtained from developer activities that can then be used as developer metrics to solve a variety of challenges. In this study, we proposed new developer metrics extracted from the issue, commit, and pull request activities of developers on GitHub. We created developer metrics from the individual activities and combined certain activities according to some common traits. To evaluate these metrics, we created an item-based project recommendation system. In order to validate this system, we calculated the similarity score using two methods and assessed top-n hit scores using two different approaches. The results for all scores with these methods indicated that the most successful metrics were binary_issue_related, issue_commented, binary_pr_related, and issue_opened. To verify our results, we compared our metrics with another metric generated from a very similar study and found that most of our metrics gave better scores that metric. In conclusion, the issue feature is more crucial for GitHub compared with other features. Moreover, commenting activity in projects can be equally as valuable as code contributions. The most of binary metrics that were generated, regardless of the number of activities, also showed remarkable results. In this context, we presented improvable and noteworthy developer metrics that can be used for a wide range of open-source software development challenges, such as user characterization, project recommendation, and code review assignment.

ОГЛЯД МЕТРИК ТА АНАЛІЗ ЇХНЬОГО ВИКОРИСТАННЯ ПРИ ОЦІНЦІ ЕФЕКТИВНОСТІ IT-ПРОЕКТІВ

У статті виділені основні показники при оцінці ефективності IT-проектів. Зібрані дані про сучасні метрики, які успішно використовують великі IT-компанії. Розкривається поняття "ієрархія метрик", показаний зв'язок з ієрархією методології BOCR, що використовується для оцінки ефективності.Проведений огляд сучасних метрик в IT-галузі та розглянута проблема підбору успішних метрик, які в сумі будуть найбільш повною мірою описувати основні аспекти ефективності IT-проектів.Запропоновані метрики за узагальненими показниками, які успішно використовують у практиці IT-компанії. Приводяться аргументи щодо використання запропонованих метрик в реальному менеджменті в IT-компаніях.

ARTIFICIAL NEURAL NETWORKS APPLIED IN FOREST BIOMETRICS AND MODELING: STATE OF THE ART (JANUARY/2007 TO JULY/2018)

Artificial Intelligence has been an important support tool in different spheres of activity, enabling the knowledge aggregation, the process optimization and the application of methodologies capable of solving complex real problems. Although focusing on a wide space of successful metrics, the Artificial Neural Networks (ANN) approach, a technique similar to the central nervous system, has been gaining notoriety and relevance with regard to the classification of standards, intrinsic parameter estimates, remote sense, data mining and other possibilities. This article aims to develop a systematic review, involving some bibliometric aspects, aimed at detecting the application of ANNs in the field of Forest Engineering, particularly in the prognosis of the essential parameters to the forest inventory, analyzing the construction of the scopes, implementation of networks (type – classification), the software used and complementary techniques. From the 1,140 articles collected in three search databases (Science Direct, Scopus and Web of Science), 43 articles underwent such analyzes. The results show that the number of works within this scope has been increasing continuously, and 32% of the analyzed articles predict the final total marketable volume; 78% made use of the Multilayer Perceptron Networks (MLP, Multilayer Perceptron) and 63% were from Brazilian researchers.

CLIA validation workflow of a novel tumor cell isolation platform.

e23161 Background: DEPArray™ technology is based on Dielectrophoresis (DEP). High quality image-based cell selection enables users to identify, isolate and recover intact specific individual rare cells of interest from complex, heterogeneous tissues such as live or fixed cell suspensions. Here we demonstrate a validation workflow for application of molecular sequencing technologies and FISH downstream of rare cell recovery. Methods: FFPE scrolls (n = 93) were dissociated and stained with markers to distinguish putative stromal and tumor populations. DNA integrity was tested using a qPCR technique to predict the amount of DNA required for a successful downstream sequencing metrics. DEPArray™ recovered stromal and tumor cells underwent NGS analysis on an Illumina platform based DEPArray™ OncoSeek Panel. Separately, recovered stromal and tumor cells were also utilized for targeted HER2 FISH. CellSearch® cartridges (19 Breast and 5 Bladder Cancers) were processed through the DEPArray to isolate WBCs (CK-/CD45+/DAPI+; n = 30), CTCs (CK+/CD45-/DAPI+; n = 33) and atypical (CK-/CD45-/DAPI+; n = 47) cells. Recovered cells underwent whole-genome amplification with Ampli1™ WGA and quality controlled by Ampli1 QC. Ampli1 LowPass kit was then used to prepare NGS libraries for absolute CNA profiling by low-pass WGS. Results: Reproducibility and reliability was reported as 100% for instrument performance. For FFPE, the OncoSeek panel simultaneously detected SNPs, indels and CNAs of 63actionable and oncology relevant genes. For FISH, 95% concordance with conventional HER2 results was observed. For CTCs, Ampli1 LowPass detected copy number gains/losses confirming tumor origin of the CK+ cells. A portion of non-conventional cells also demonstrated copy number alterations consistent with tumor origin. Conclusions: The isolation of pure single or pooled tumor cells with the DEPArray™ technology can be used as a method to improve downstream MDx analysis using different techniques, thus to inform treatment decisions and provide valuable prognostic information.

Going to scale: dissemination and implementation of EBPs across large publicly-funded health care systems - initiatives targeting system-level change

This panel will examine strategies and outcomes across three initiatives aimed at translating evidence-based care models and practices in the treatment of low-income, disadvantaged patients with behavioral health and other complex conditions at scale, across states and cities. The initiatives include translation across states of measurement-driven QI initiatives in mental health in the six-state MEDNET consortium; translation across cities of the care coordination model developed for low-income super-utilizers by Brenner and colleagues in Camden; and the implementation across 91 California sites of the Full Service Partnership (FSP) model for persons with severe mental illness who are homeless or at risk of homelessness. The discussion will initially be framed by a brief overview from the organizer/discussant placing the three interventions within an overall conceptual model. Following the presentations, comments from the discussant will compare and contrast the effectiveness of the alternative strategies utilized across care processes and sites. Common themes across the three initiatives will be discussed, in terms of types of barriers and facilitators experienced in the process of dissemination at scale. We will discuss the barriers and facilitators experienced with respect to sustainability of the interventions beyond the initial project period; e.g., variations in successful incorporation metrics for evidence-based practices into ongoing management processes in public programs, and in incorporating guidelines for the use of evidence-based practices into state and health plan clinical guidelines, such as the Texas foster care psychotropic prescribing parameters and clinical guidelines promulgated by health plans across sites. We will discuss the important distinction between the types of barriers and facilitators experienced in the translation of specific program models across sites at small to medium scale, and the different set of barriers and facilitators that are encountered in efforts to bring these translational efforts to statewide and other large-scale systems with sustained impact. Meaningful antipsychotic metrics for state medicaid programs: strategies for sustainable change and quality mental health care Safe and effective use of antipsychotic medications across multiple populations, often off-label, presents many challenges for Medicaid programs. We will discuss the experience of a 6-state consortium, the Medicaid Mental Health Network for Evidence Based Care (MEDNET), in developing consensus quality metrics for safe and judicious antipsychotic use and translating measurement-driven QI initiatives at scale in large statewide populations. The presentation of barriers and facilitators experienced in development and use of metrics and translation of interventions across states with different policy and organizational environments.

Quantifying surface albedo and other direct biogeophysical climate forcings of forestry activities.

By altering fluxes of heat, momentum, and moisture exchanges between the land surface and atmosphere, forestry and other land-use activities affect climate. Although long recognized scientifically as being important, these so-called biogeophysical forcings are rarely included in climate policies for forestry and other land management projects due to the many challenges associated with their quantification. Here, we review the scientific literature in the fields of atmospheric science and terrestrial ecology in light of three main objectives: (i) to elucidate the challenges associated with quantifying biogeophysical climate forcings connected to land use and land management, with a focus on the forestry sector; (ii) to identify and describe scientific approaches and/or metrics facilitating the quantification and interpretation of direct biogeophysical climate forcings; and (iii) to identify and recommend research priorities that can help overcome the challenges of their attribution to specific land-use activities, bridging the knowledge gap between the climate modeling, forest ecology, and resource management communities. We find that ignoring surface biogeophysics may mislead climate mitigation policies, yet existing metrics are unlikely to be sufficient. Successful metrics ought to (i) include both radiative and nonradiative climate forcings; (ii) reconcile disparities between biogeophysical and biogeochemical forcings, and (iii) acknowledge trade-offs between global and local climate benefits. We call for more coordinated research among terrestrial ecologists, resource managers, and coupled climate modelers to harmonize datasets, refine analytical techniques, and corroborate and validate metrics that are more amenable to analyses at the scale of an individual site or region.

Factors Determining Long-term Success of a Measurement Program: An Industrial Case Study

Introducing measurement programs into organizations is a lengthy process aected by organizational and technical constraints. There exist several aspects that determine whether a measurement program has the chances of succeeding, like management commitment or existence of proper tool support. The establishing of a program, however, is only a part of the success. As organizations are dynamic entities, the measurement programs should constantly be maintained and adapted in order to cope with changing needs of the organizations. In this paper we study one of the measurement programs at Ericsson AB in Sweden and as a result we identify factors determining successful adoption and use of the measurement program. The results of our research in this paper are intended to support quality managers and project managers in establishing and maintaining successful metrics programs.

Metrics collection for process knowledge—a practitioners' guide

AbstractThis paper describes a number of experiences of gathering metrics to supply the knowledge needed for decision making and process management in an industrial environment. The experiences are summarised as guidelines in the form of a list of ‘Do’s and ‘Do not’s for collecting and using metrics for this purpose. While the paper is clearly anecdotal in nature, these guidelines should, nevertheless, allow other managers to draw on the experience of the authors to put a successful metrics programme in place to provide knowledge for decision making and process improvement.The paper concludes, however, that there are limitations that exist for metrics programmes implemented, even if the guidelines are followed. The metrics cannot always offer evidence that any particular action or technique has given a better project management methodology. However, when used with other investigations they can give a better knowledge and understanding of processes that should enable more effective management. Copyright © 2006 John Wiley & Sons, Ltd.

Metrics and management: applying the results of the balanced scorecard

The paper reviews the balanced scorecard's first year of operation at the University of Virginia Library. It focuses on outcomes, offering answers to some basic questions. How were the results used? What was done to address shortfalls? Were successful metrics and their targets revised? What changes were made to the overall suite of metrics? Has the scorecard improved the organization? The paper serves as a case study in the use of the balanced scorecard in an academic library, and informs the audience of the strengths and weaknesses of this management tool.

Cultivation and engineering of a software metrics program

Abstract. This paper reports from a case study of an organization that implements a software metrics program to measure the effects of its improvement efforts. The program measures key indicators of all completed projects and summarizes progress information in a quarterly management report. The implementation turns out to be long and complex, as the organization is confronted with dilemmas based on contradictory demands and value conflicts. The process is interpreted as a combination of a rational engineering process in which a metrics program is constructed and put into use, and an evolutionary cultivation process in which basic values of the software organization are confronted and transformed. The analysis exemplifies the difficulties and challenges that software organizations face when bringing known principles for software metrics programs into practical use. The article discusses the insights gained from the case in six lessons that may be used by Software Process Improvement managers in implementing a successful metrics program.

An information theory-based approach to quantifying the contribution of a software metric

Competitive pressures are forcing many companies to aggressively pursue software quality improvement based on software complexity metrics. A metrics database is often the key to a successful ongoing software metrics program. Contel had a successful metrics program that involved project-level metrics databases and planned a corporate level database. The U.S. Army has established a minimum set of metrics for Army software development and maintenance covering the development process, software quality, and software complexity. This program involves a central Army-wide metrics database and a validation program. In light of the importance of corporate metrics databases and the prevalence of multicolliner metrics, we define the contribution of any proposed metric in terms of the measured variation, irrespective of the metric's usefulness in quality models. This is of interest when full validation is not practical. We review two approaches to assessing the contribution of a new software complexity metric to a metrics database and present a new method based on information theory. The method is general and does not presume any particular set of metrics. We illustrate this method with three case studies, using data from full-scale operational software systems. The new method is less subjective than competing assessment methods.

IMPLEMENTING A SUCCESSFUL METRICS PROGRAM

AbstractIn today's highly competitive business environment, with its tight budgets and pressure for continuous improvement and reengineering, qualitative improvements are no longer adequate for an organization to be competitive in the marketplace. Improvements need to be measurable, which requires the availability of quantified baselines and metrics to support the quantification. This paper describes a framework for collection and utilization of metrics within an organization involved in the engineering, development, and deployment of complex systems; the types of metrics that are needed to measure the effectiveness of an organization's processes; and a proposed framework for their utilization at the tactical and strategic levels, including their roles in process improvement and reengineering. This paper also discusses commonly encountered pitfalls and guidelines for implementing a successful metrics program.