Subset Of Metrics Research Articles

Metrics for Dataset Demographic Bias: A Case Study on Facial Expression Recognition.

Demographic biases in source datasets have been shown as one of the causes of unfairness and discrimination in the predictions of Machine Learning models. One of the most prominent types of demographic bias are statistical imbalances in the representation of demographic groups in the datasets. In this article, we study the measurement of these biases by reviewing the existing metrics, including those that can be borrowed from other disciplines. We develop a taxonomy for the classification of these metrics, providing a practical guide for the selection of appropriate metrics. To illustrate the utility of our framework, and to further understand the practical characteristics of the metrics, we conduct a case study of 20 datasets used in Facial Emotion Recognition (FER), analyzing the biases present in them. Our experimental results show that many metrics are redundant and that a reduced subset of metrics may be sufficient to measure the amount of demographic bias. The article provides valuable insights for researchers in AI and related fields to mitigate dataset bias and improve the fairness and accuracy of AI models.

Selecting optimal software code descriptors-The case of Java.

Over the last 25 years, a considerable proliferation of software metrics and a plethora of tools have emerged to extract them. While this is indeed positive concerning the previous situations of limited data, it still leads to a significant problem arising both from a theoretical and a practical standpoint. From a theoretical perspective, several metrics are likely to result in collinearity, overfitting, etc. From a practical perspective, such a set of metrics is difficult to manage and companies, especially small ones, may feel overwhelmed and unable to select a viable subset of them. Still, so far it has not been fully understood what is a viable subset of metrics suitable to properly manage software projects and products. In this paper, we attempt to address this issue. We focus on the case of programs written in Java and we consider classes and methods. We use Sammon error as a measure of the similarity of metrics. Utilizing both Particle Swarm Optimization and Genetic Algorithm, we adapted a method for the identification of a viable subset of such metrics that could solve the mentioned problem. Furthermore, we experiment with our approach on 800 projects coming from GitHub and validate the results on 200 projects. With the proposed method we got optimal subsets of software engineering metrics. These subsets gave us low values of Sammon error at more than 70% at class and method levels on a validation dataset.

Access to Care for Adolescents and Young Adults With Cancer in the United States: State of the Literature.

Access to care remains a persistent challenge for adolescents and young adults (AYAs) with cancer. We review key findings in the science to date. (1) Location of care matters. There is survival benefit for AYAs treated either at a pediatric center or site with special status (eg, Children's Oncology Group, National Cancer Institute [NCI]-designated Comprehensive Cancer Center). (2) Socioeconomic status and insurance require further investigation. Medicaid expansion has had a moderate effect on AYA outcomes. The dependent care expansion benefit has come largely from improvements in coverage for younger populations whose parents have insurance, while some subgroups likely still face insurance gaps. (3) Clinical trial enrollment remains poor, but access may be improving. Numerous barriers and facilitators of clinical trial enrollment include those that are system level and patient level. NCI has established several initiatives over the past decade to improve enrollment, and newer collaboratives have recently brought together multidisciplinary US teams to increase clinical trial enrollment. (4) Effective AYA programs require provider and system flexibility and program reflection. With flexibility comes a need for metrics to assess program effectiveness in the context of the program model. Centers treating AYAs with cancer could submit a subset of metrics (appropriate to their program and/or services) to maintain their status; persistence would require an entity with staying power committed to overseeing the metrics and the system. Substantial clinical and biological advances are anticipated over the next 20 years that will benefit all patients with cancer. In parallel, it is crucial to prioritize research regarding access to health care and cancer care delivery; only with equitable access to care for AYAs can they, too, benefit from these advances.

Effectiveness of regional diffusion MRI measures in distinguishing multiple sclerosis abnormalities within the cervical spinal cord.

Multiple sclerosis (MS) is an inflammatory disorder of the central nervous system. Although conventional magnetic resonance imaging (MRI) is widely used for MS diagnosis and clinical follow-up, quantitative MRI has the potential to provide valuable intrinsic values of tissue properties that can enhance accuracy. In this study, we investigate the efficacy of diffusion MRI in distinguishing MS lesions within the cervical spinal cord, using a combination of metrics extracted from diffusion tensor imaging and Ball-and-Stick models. We analyzed spinal cord data acquired from multiple hospitals and extracted average diffusion MRI metrics per vertebral level using a collection of image processing methods and an atlas-based approach. We then performed a statistical analysis to evaluate the feasibility of these metrics for detecting lesions, exploring the usefulness of combining different metrics to improve accuracy. Our study demonstrates the sensitivity of each metric to underlying microstructure changes in MS patients. We show that selecting a specific subset of metrics, which provide complementary information, significantly improves the prediction score of lesion presence in the cervical spinal cord. Furthermore, the Ball-and-Stick model has the potential to provide novel information about the microstructure of damaged tissue. Our results suggest that diffusion measures, particularly combined measures, are sensitive in discriminating abnormal from healthy cervical vertebral levels in patients. This information could aid in improving MS diagnosis and clinical follow-up. Our study highlights the potential of the Ball-and-Stick model in providing additional insights into the microstructure of the damaged tissue.

On the relationship between source-code metrics and cognitive load: A systematic tertiary review

The difficulty of software development tasks depends on several factors including the characteristics of the underlying source-code. These characteristics can be captured and measured using source-code metrics, which, in turn, can provide indications about the difficulty of the source-code. From a cognitive perspective, this difficulty is due to an increase in developers’ cognitive load, which can be estimated using psycho-physiological measures. Based on these measures, a handful of studies investigated the relationship between source-code metrics and cognitive load. For most of the metrics, such a relationship could not be established. While these studies used a small subset of metrics, the literature comprises hundreds of other metrics. Despite the existing reviews surveying these metrics, a consolidated overview is still needed to understand their properties and leverage their potential to align with cognitive load. This need is addressed in this paper through a Systematic Tertiary Review (STR) covering the full spectrum of source-code metrics, studying their properties and investigating their potential relationship to cognitive load. The outcome of this STR is intended to guide practitioners in choosing appropriate metrics, set the grounds for conceptualizing the relationship between source-code metrics and cognitive load and raise new research challenges for the future.

Using Collective Metrics to Assess Team Dynamics and Performance in eSports

A challenge posed by virtual teams is monitoring team interactions remotely. Research with field-based soccer teams provides evidence that measures of collective behavior can be used to assess the dynamics of sports teams. Collective behaviors calculated using the spatial characteristics of teammates as they moved across the field have been found to vary by the state of the soccer match, including ball possession and proximity to a goal. The present study examined whether similar effects were observed with collective metrics calculated from players of a car-soccer eSport video game. A set of matches were retrieved and used to calculate collective behavior metrics based on the placement of teammates within a virtual arena. A subset of metrics varied by team location and ball possession, aligning with and extending previous field-based soccer research, and correlated with team performance. This suggests that collective behaviors can be used to assess aspects of team dynamics within virtual environments.

Adaptive Performance Analysis in IoT Platforms

In this paper, we consider the problem of identifying multiple bottlenecks (a.k.a bottleneck analysis) in IoT Service Platforms. For QoS-constrained applications, IoT Platforms have grown in complexity with non-stationary workloads and inter-task dependencies created by data flows crossing the platform’s nodes. These factors create multiple simultaneous “bottlenecks” (a bottleneck expresses overload in terms of request processing time on a given node, and contributes to QoS degradation). Multi-bottlenecks are non-trivial to analyze since they may escape typical assumptions made in classic performance analysis, such as analysis based on queuing theory models. Solving this analysis problem requires real-time collection and analysis of data that can be massive, and as a result, induce negative impacts on the performance of the NFV-based IoT Platform (NIP) (e.g., use of bandwidth, computing resource, and storage resource). Therefore, it needs to be adapted to the strict minimum allowing effective analysis. We build an adaptive performance analysis method that optimizes bottlenecks’ identification for a monitoring overhead budget associated with the different available metrics. Instead of systematically collecting all the NIP metrics, the proposed process determines the best subset of metrics to consider for the efficiency of the performance analysis. The conducted experiments on a practical use case show that the proposed method exhibited high performances of the bottleneck analysis process, in the presence of different bottleneck types and durations, with very few false positives and false negatives.

Comparisons of Filter, Wrapper and Embedded-Based Feature Selection Techniques for Consistency of Software Metrics Analysis

Identifying and selecting the most consistent subset of metrics which improves the performance of software defect prediction model is paramount but challenging problem as it receives little attention in literature. The current research aimed at investigating the consistency of subsets of metrics that are produced by embedded feature selection techniques. Ten (10) feature selection techniques used from the families of filter and wrapper-based feature selection techniques commonly used in the defect prediction domain. Ten (10) publicly available defect datasets were studied which span both proprietary and open source domains. SVM-RFE-RF presented 42-93% consistent metrics across datasets. While the prior study on non-Embedded produced 56.5% consistent metrics at median. SVM-RFE-LF approach of Embedded Feature Selection Technique produced 54-80% consistent metrics across datasets and 42.5% at median. To state the purpose of tittle has been achieved Embedded based Feature Selection Techniques produced most efficient consistent subset selection across the entire datasets and amongst the feature selection techniques as compared with counterpart filter and wrapper-based feature selection techniques

Evaluating algorithms for anomaly detection in satellite telemetry data

Detecting anomalies in telemetry data captured on-board a spacecraft is critical to ensure its safe operation. Although there exist various techniques for automatically detecting point, contextual, and collective anomalies from time-series data, quantifying their performance remains under-researched. In this paper, we thoroughly validate our approach for the task of anomalous event detection that is built upon a two-stage technique, in which the telemetry signal is predicted using a long short-term memory network based on the historical data, and then the prediction is compared with the actual (captured) data. If the difference between those two is sufficiently large, we can infer that an anomalous event has happened. To evaluate the capabilities of such detection techniques over the simulated and benchmark time-series data, we investigate a set of commonly used metrics obtained for a range of anomaly detection approaches, and present their shortcomings, especially related to their inability of capturing the temporal aspects of the detectors. We tackle this issue by introducing new quality metrics which enable us to objectively verify if the detectors can timely spot the anomalies in sequential data. The experimental study showed that inferring the conclusions based on a subset of metrics can lead to biased observations, as the best algorithms determined based on the overlap metrics, including the Dice coefficient, do not necessarily correspond to the algorithms that offer the fastest detection. Finally, we discuss the Antelope Toolbox—our software tool for simulating nominal telemetry data of given characteristics, alongside well-defined anomalous events, and to perform the quantitative and qualitative analysis of anomaly detection algorithms over such simulated events.

Generic homeomorphisms have full metric mean dimension

AbstractWe prove that for $C^0$ -generic homeomorphisms, acting on a compact smooth boundaryless manifold with dimension greater than one, the upper metric mean dimension with respect to the smooth metric coincides with the dimension of the manifold. As an application, we show that the upper box dimension of the set of periodic points of a $C^0$ -generic homeomorphism is equal to the dimension of the manifold. In the case of continuous interval maps, we prove that each level set for the metric mean dimension with respect to the Euclidean distance is $C^0$ -dense in the space of continuous endomorphisms of $[0,1]$ with the uniform topology. Moreover, the maximum value is attained at a $C^0$ -generic subset of continuous interval maps and a dense subset of metrics topologically equivalent to the Euclidean distance.

The use of web analytics combined with other data streams for tailoring online vaccine safety information at global level: The Vaccine Safety Net’s web analytics project

The Vaccine Safety Net’s Web Analytics Project (VSN-WAP) was launched in October 2017 to monitor the behavior of users visiting websites belonging to the VSN, a global network of websites providing science-based information on vaccine safety. Participating websites could provide web metrics in two ways: through a Google Analytics (GA) script, which automatically forwarded metrics to a central account and through manual input (MI) of a reduced subset of metrics (Sessions, Page Views, New Users, Bounce Rate, Views/Session and Average Session Duration), which were pooled with the metrics obtained through GA. Additional metrics were obtained from websites providing data through Google Analytics (Country, Age, Sex, Device). We report results from February 2018 to March 2019. In March 2019, 32 websites were participating in the project (21 through GA, 11 through MI). From February 2018 to March 2019 we recorded 22,471,535 sessions, with 38,307,349 page views. Sessions, New Users and Page views progressively increased, Views/Session, Bounce Rate and Average Session Duration remained stable. Most users were female (68%) and belonged to the 25–34 age range (37%), followed by 35–44 (22%) and 18–24 (19%). Fifty-four percent of users connected from a mobile device, 42% from a desktop and 4% from a tablet.Digital media monitoring techniques can provide insights on the characteristics of users with a specific interest in vaccines. These data can be exploited to improve the performance of websites providing information on vaccines to the general public.

A treatment planning study of combined carbon ion-beam plus photon intensity-modulated radiotherapy.

Combined photon intensity-modulated radiotherapy (IMRT) and sequential dose-escalated carbon ion beam therapy (IBT) is a technically advanced treatment option for head and neck malignancies. We proposed and evaluated an integrated planning strategy as opposed to an established and largely separated planning workflow. Ten patients with representative malignancies of the head and neck region underwent combined carbon-photon radiotherapy (RT) in our facilities. Clinical plans were created according to the separated workflow with independent optimization stages for both modalities. Experimental plans incorporated the existing carbon IBT dose distribution into the optimization stage of a step-and-shoot photon IMRT (bias dose planning). Cumulative dose distributions showed statistically significant differences between the two planning strategies and were predominantly in favor of the integrated approach. As such, target irradiation was generally maintained or even improved in a subset of metrics, while normal tissue sparing was widely enhanced; for instance, in the ipsilateral temporal lobe with median Dmean of -16% (p<0.001). Maximum doses D1% (with adjustment for different fractionation) fell below thresholds for toxicity risk in a minority of instances, where they were previously exceeded. Integral dose did not differ significantly. Our findings indicate that combination planning of carbon-photon RT for head and neck malignancies may benefit from a proposed bias dose method, yielding favorable dose distribution characteristics and a streamlined planning workflow with fewer plan revisions. Further research is necessary to validate these observations in terms of robustness and their potential for higher tumor control.

Robust HDR image quality assessment using combination of quality metrics

We propose a framework to combine various quality metrics using a full reference approach for High Dynamic Range (HDR) Image quality assessment (IQA). We combine scores from metrics exclusively designed for different applications such as HDR, Standard Dynamic Range (SDR) and color difference measures, in a non-linear manner using machine learning (ML) approaches with weights determined during an offline training process. We explore various ML techniques and find that support vector machine regression and gradient boosting regression trees are effective. To improve performance and reduce complexity, we use the back-tracking based Sequential Floating Forward Selection technique during training to include a subset of metrics from a list of quality metrics in our model. We evaluate the performance on five publicly available calibrated HDR databases with different types of distortion (including different types of compression, Gaussian noise, gamut mismatch, chromatic distortions and so on) and demonstrate improved performance using our method as compared to several existing IQA metrics. We perform extensive statistical analysis to demonstrate significant improvement over existing approaches and show the generality and robustness of our approach using cross-database validation.

Visual Multi-Metric Grouping of Eye- Tracking Data.

We present an algorithmic and visual grouping of participants and eye-tracking metrics derived from recorded eye-tracking data. Our method utilizes two well-established visualization concepts. First, parallel coordinates are used to provide an overview of the used metrics, their interactions, and similarities, which helps select suitable metrics that describe characteristics of the eye-tracking data. Furthermore, parallel coordinates plots enable an analyst to test the effects of creating a combination of a subset of metrics resulting in a newly derived eye-tracking metric. Second, a similarity matrix visualization is used to visually represent the affine combination of metrics utilizing an algorithmic grouping of subjects that leads to distinct visual groups of similar behavior. To keep the diagrams of the matrix visualization simple and understandable, we visually encode our eyetracking data into the cells of a similarity matrix of participants. The algorithmic grouping is performed with a clustering based on the affine combination of metrics, which is also the basis for the similarity value computation of the similarity matrix. To illustrate the usefulness of our visualization, we applied it to an eye-tracking data set involving the reading behavior of metro maps of up to 40 participants. Finally, we discuss limitations and scalability issues of the approach focusing on visual and perceptual issues.

A Framework for Software Defect Prediction and Metric Selection

Automated software defect prediction is an important and fundamental activity in the domain of software development. However, modern software systems are inherently large and complex with numerous correlated metrics that capture different aspects of the software components. This large number of correlated metrics makes building a software defect prediction model very complex. Thus, identifying and selecting a subset of metrics that enhance the software defect prediction method’s performance are an important but challenging problem that has received little attention in the literature. The main objective of this paper is to identify significant software metrics, to build and evaluate an automated software defect prediction model. We propose two novel hybrid software defect prediction models to identify the significant attributes (metrics) using a combination of wrapper and filter techniques. The novelty of our approach is that it embeds the metric selection and training processes of software defect prediction as a single process while reducing the measurement overhead significantly. Different wrapper approaches were combined, including SVM and ANN, with a maximum relevance filter approach to find the significant metrics. A filter score was injected into the wrapper selection process in the proposed approaches to direct the search process efficiently to identify significant metrics. Experimental results with real defect-prone software data sets show that the proposed hybrid approaches achieve significantly compact metrics (i.e., selecting the most significant metrics) with high prediction accuracy compared with conventional wrapper or filter approaches. The performance of the proposed framework has also been verified using a statistical multivariate quality control process using multivariate exponentially weighted moving average. The proposed framework demonstrates that the hybrid heuristic can guide the metric selection process in a computationally efficient way by integrating the intrinsic characteristics from the filters into the wrapper and using the advantages of both the filter and wrapper approaches.

A consistent conceptual framework for applying climate metrics in technology life cycle assessment

Comparing the potential climate impacts of different technologies is challenging for several reasons, including the fact that any given technology may be associated with emissions of multiple greenhouse gases when evaluated on a life cycle basis. In general, analysts must decide how to aggregate the climatic effects of different technologies, taking into account differences in the properties of the gases (differences in atmospheric lifetimes and instantaneous radiative efficiencies) as well as different technology characteristics (differences in emission factors and technology lifetimes). Available metrics proposed in the literature have incorporated these features in different ways and have arrived at different conclusions. In this paper, we develop a general framework for classifying metrics based on whether they measure: (a) cumulative or end point impacts, (b) impacts over a fixed time horizon or up to a fixed end year, and (c) impacts from a single emissions pulse or from a stream of pulses over multiple years. We then use the comparison between compressed natural gas and gasoline-fueled vehicles to illustrate how the choice of metric can affect conclusions about technologies. Finally, we consider tradeoffs involved in selecting a metric, show how the choice of metric depends on the framework that is assumed for climate change mitigation, and suggest which subset of metrics are likely to be most analytically self-consistent.

Software maintainability prediction using hybrid neural network and fuzzy logic approach with parallel computing concept

In present day scenario, majority of software companies use object-oriented concept to develop software systems as it enables effective design, development, testing and maintenance, in addition to the optimal characterization of the software system. With the increase in number of these software systems, their effective maintenance aspect becomes very important day by day. In this study, Neuro-Fuzzy approach: hybrid neural network and fuzzy logic approach has been considered to develop a maintainability model using ten different object-oriented static source code metrics as input. This method is applied on maintainability data of two commercial software products such as UIMS and QUES. Rough set analysis (RSA) and principal component analysis (PCA) are used to select suitable set of metrics from the ten metrics employed to improve performance of maintainability prediction model. From experimental results, it is observed that Neuro-Fuzzy model can effectively predict the maintainability of object-oriented software systems. After implementing parallel computing concept, it is observed that the training time gets reduced to a significant amount when the number of computing nodes were increased. Further it is observed that selected subset of metrics using feature selection techniques i.e., PCA, and RSA was able to predict maintainability with higher accuracy.

Classification of certain asymptotically AdS space-times with Ricci-flat boundary

We classify solutions to Einstein's equations in AdS with Ricci-flat boundary metric and with covariantly constant boundary stress tensor, which in general is not diagonalizable, i.e. it does not admit a reference frame. New solutions are found, and in the context of the AdS/CFT duality they should describe a boundary QFT in certain non-equilibrium steady states. Further imposing the absence of scalar curvature singularities leads to a subset of metrics that can be seen as null deformations of AdS or of the AdS soliton. We also outline the procedure of solving the equations when a scalar is coupled to the metric, which holographically leads to non-Lorentz-invariant RG flows.

How livestock and flooding mediate the ecological integrity of working forests in Amazon River floodplains.

The contribution of working forests to tropical conservation and development depends upon the maintenance of ecological integrity under ongoing land use. Assessment of ecological integrity requires an understanding of the structure, composition, and function and major drivers that govern their variability. Working forests in tropical river floodplains provide many goods and services, yet the data on the ecological processes that sustain these services is scant. In flooded forests of riverside Amazonian communities, we established 46 0.1-ha plots varying in flood duration, use by cattle and water buffalo, and time since agricultural abandonment (30-90 yr). We monitored three aspects of ecological integrity (stand structure, species composition, and dynamics of trees and seedlings) to evaluate the impacts of different trajectories of livestock activity (alleviation, stasis, and intensification) over nine years. Negative effects of livestock intensification were solely evident in the forest understory, and plots alleviated from past heavy disturbance increased in seedling density but had higher abundance of thorny species than plots maintaining low activity. Stand structure, dynamics, and tree species composition were strongly influenced by the natural pulse of seasonal floods, such that the defining characteristics of integrity were dependent upon flood duration (3-200 d). Forests with prolonged floods ≥ 140 d had not only lower species richness but also lower rates of recruitment and species turnover relative to forests with short floods <70 d. Overall, the combined effects of livestock intensification and prolonged flooding hindered forest regeneration, but overall forest integrity was largely related to the hydrological regime and age. Given this disjunction between factors mediating canopy and understory integrity, we present a subset of metrics for regeneration and recruitment to distinguish forest condition by livestock trajectory. Although our study design includes confounded factors that preclude a definitive assessment of the major drivers of ecological change, we provide much-needed data on the regrowth of a critical but poorly studied ecosystem. In addition to its emphasis on the dynamics of tropical wetland forests undergoing anthropogenic and environmental change, our case study is an important example for how to assess of ecological integrity in working forests of tropical ecosystems.

Local and gauge invariant observables in gravity

It is well known that general relativity (GR) does not possess any non-trivial local (in a precise standard sense) and diffeomorphism invariant observable. We propose a generalized notion of local observables, which retain the most important properties that follow from the standard definition of locality, yet is flexible enough to admit a large class of diffeomorphism invariant observables in GR. The generalization comes at a small price—that the domain of definition of a generalized local observable may not cover the entire phase space of GR and two such observables may have distinct domains. However, the subset of metrics on which generalized local observables can be defined is in a sense generic (its open interior is non-empty in the Whitney strong topology). Moreover, generalized local gauge invariant observables are sufficient to separate diffeomorphism orbits on this admissible subset of the phase space. Connecting the construction with the notion of differential invariants gives a general scheme for defining generalized local gauge invariant observables in arbitrary gauge theories, which happens to agree with well-known results for Maxwell and Yang–Mills theories.