Reliability Metrics Research Articles

KovaaK's aim trainer as a reliable metrics platform for assessing shooting proficiency in esports players: a pilot study.

Esports research lacks game-based metrics platforms appropriate for adequately capturing esports performance. The aim of this pilot study was to assess the reliability of the KovaaK's first-person shooter (FPS) aim trainer as a metrics platform for assessing shooting proficiency in esports players. Ten FPS esports players completed two identical experimental trials (T) separated by three to five days. Each trial included four rounds (R) of testing, evaluating four shooting tasks: Micro Flicking, Macro Flicking, Strafe Tracking, and Wall Peeking. Reliability of performance outcomes (e.g., accuracy, headshot accuracy, hits per second, and total shots hit) were assessed using the intraclass correlation coefficient (ICC) and their 95% confidence intervals (CI), and significant differences were identified using repeated-measures analysis of variance (RM-ANOVA). Results indicated excellent, or good to excellent reliability for all outcome variables with the ICC estimates ranging between 0.947-0.995, with lower and upper bound 95% CIs ranging between 0.876-0.988, and 0.984-0.999, respectively. Significant improvements were seen between experimental trials in the Macro Flicking task for accuracy (p = .005) and hits per second (p = .009) only. Significant interactions between trial and round were identified in the Micro Flicking task for accuracy (p = .006), with post hoc analysis showing accuracy was significantly higher in T1R1 compared to T2R1 (87.74 ± 3.13 vs. 85.99 ± 3.05, respectively, p = .02), and in T2R4 compared to T2R2 (87.99 ± 2.89 vs. 84.70 ± 4.25, respectively, p = .049). Significant interactions were also identified in the Strafe Tracking task for headshot accuracy (p = .002), with post hoc analysis showing headshot accuracy was significantly higher in T1R2 compared to T2R2 (78.48 ± 8.15 vs. 76.79 ± 12.16, respectively, p = .003), and in T1R2 compared to T1R1 (78.48 ± 8.15 vs. 73.68 ± 17.94, respectively, p = .023). In summary, this study demonstrates that KovaaK's provides a reliable metrics platform for assessing shooting proficiency in esports, however, some variability in performance was observed.

GROUNDNUT CROP PEST DETECTION AND CLASSIFICATION USING COMPREHENSIVE DEEP-LEARNING MODELS

Pests pose a significant threat to crops, leading to substantial economic losses and decreased food production. Early detection and accurate classification of pests in crops are crucial for effective pest management strategies. In this study, we propose a method for pest detection and classification in groundnut crops using deep learning models. In this research, we compare the performance of three deep learning models, namely Custom CNN [proposed], LeNet-5, and VGG-16, for groundnut pest detection and classification. A comprehensive dataset containing images of diverse groundnut crop pests, including thrips, aphids, armyworms, and wireworms, from the IP102 dataset was utilized for model evaluation. The performance is evaluated using reliability metrics such as accuracy and loss. These findings demonstrate the utility of deep learning models for reliable pest classification of groundnut crops. The Custom CNN [proposed] model demonstrates high training accuracy but potential overfitting, while the VGG-16 model performs well on both training and test data, showcasing its ability to generalize. The models’ accuracy in predicting pest species underscores their capability to capture and utilize visual patterns for precise classification. These findings underscore the potential of deep learning models, particularly the VGG-16 model, for pest detection and classification in groundnut crops. The knowledge gained from this study can contribute to the development of practical pest management strategies and aid in maintaining crop health and productivity. Further analysis and comparisons with other models are recommended to comprehensively evaluate the competitiveness and suitability of deep learning models in real-world applications of pest detection and classification in agricultural settings.

A fine-grained robust performance diagnosis framework for run-time cloud applications

To maintain the required service quality of time-critical cloud applications, operators must continuously monitor their runtime status, detect potential performance anomalies, and diagnose the root causes of these anomalies effectively. However, existing performance diagnosis methods face challenges such as the need for high-quality labeled data, the low reusability and robustness of performance anomaly detection models, and the absence of real-time fine-grained root cause localization. These challenges make fixing performance issues quickly and developing effective adaptation decisions difficult. We provide a Fine-grained Robust Performance Diagnosis (FIRED) framework to tackle those challenges. The framework offers a metrics selection component to filter noise and improve detection efficiency, an anomaly detection component that assembles several well-selected base models with a deep neural network, and adopts weakly supervised learning considering fewer labels exist in reality. The framework also employs a real-time, fine-grained root cause localization component to locate dependent resource metrics of performance anomalies. Our experiments show that the framework can effectively reduce data noise and achieve the best accuracy and algorithm robustness for performance anomaly detection. In addition, the framework can accurately localize the first root causes, with an average accuracy higher than 0.7 for locating the first four root cause metrics.

INTEGRATED FINANCIAL MANAGEMENT SYSTEM: ITS IMPORTANCE IN ADMINISTRATIVE MANAGEMENT IN LOCAL GOVERNMENTS IN PERU - 2023

Purpose: This study delves into the pivotal role of the Integrated Financial Administration System (SIAF) within local governments in Peru, with a focus on its influence on administrative, budgetary, and financial management. Theoretical Framework: The study is grounded in a theoretical framework that explores the essential functions and applications of the SIAF in the context of municipal affairs. Design/Methodology/Approach: Employing a quantitative research approach with a descriptive correlational design, the study actively engaged 166 participants, including authorities, managers, and collaborators from municipalities. Random surveys and interviews were conducted to gather insights from advisors and experts. Findings: The results revealed a significant consensus, with 72.29% strongly agreeing on the substantial role of the SIAF in municipal administration. An additional 70.48% endorsed strong agreement, underscoring tangible improvements in administrative management facilitated by the SIAF. Reliability analysis, as reflected in KR20 and KR21 values, further solidified the credibility of the study. Research, Practical & Social Implications: The study recommends the establishment of continuous monitoring mechanisms for the SIAF, emphasizing regular updates to align with evolving administrative, budgetary, and financial requirements. Additionally, advocating for comprehensive capacity-building and training programs for municipal personnel is crucial to enhance proficiency in utilizing the SIAF effectively. Empowering stakeholders is expected to optimize system benefits and cultivate a culture of efficient financial administration within local governments. Originality/Value: This study contributes to the existing body of knowledge by providing comprehensive insights into the indispensable role of the SIAF in local government administration, backed by rigorous quantitative analysis and reliability metrics.

Reliability of AI Algorithms in Safety Applications

The integration of Artificial Intelligence (AI) algorithms into safety-critical applications has become increasingly prevalent across various domains, including autonomous vehicles, medical diagnosis, industrial automation, and aerospace systems. These applications rely heavily on AI to make decisions that directly affect human safety, economic stability, and operational efficiency. Given the critical nature of these tasks, it is essential to rigorously assess the reliability of AI algorithms to ensure they perform consistently and accurately under all conditions. Reliability, in this context, refers to the AI system's ability to function without failure over a specific period, under defined operational conditions. In safety-critical domains, even minor errors or inconsistencies in AI decision-making can lead to catastrophic outcomes, such as traffic accidents involving autonomous vehicles, incorrect medical diagnoses leading to improper treatments, or failures in industrial processes that may cause costly downtime or even human casualties. The increasing complexity and deployment of AI technologies in these domains highlight the urgent need for a comprehensive understanding and evaluation of AI reliability. This paper provides a detailed analysis of the design considerations and methodologies for enhancing the reliability of AI algorithms. The discussion begins by exploring the underlying principles of reliability in AI systems, focusing on both theoretical and practical perspectives. We examine key factors that influence reliability, including data quality, algorithmic robustness, model interpretability, and system integration. The paper then delves into various reliability assessment techniques, such as fault tolerance mechanisms, error detection and correction methods, redundancy, and validation processes. To provide a deeper understanding of reliability in AI, we introduce mathematical models and statistical evaluation techniques that quantify reliability metrics. For instance, reliability modeling using exponential distribution, Monte Carlo simulations for probabilistic reliability analysis, and error propagation studies using Jacobian matrices are presented. We also explore the use of machine learning-specific reliability metrics, such as the Area Under the Curve (AUC) in Receiver Operating Characteristic (ROC) analysis, which helps evaluate the performance of AI in critical decision-making contexts. Furthermore, this paper addresses the current challenges and limitations in ensuring AI reliability, including computational complexity, ethical considerations, and regulatory compliance issues. We highlight the difficulties in developing AI models that can maintain their reliability across diverse and unpredictable real-world scenarios. The potential for bias, lack of transparency in AI decision-making, and difficulties in explaining complex AI models also present significant hurdles that need to be addressed to enhance reliability. The findings and methodologies discussed in this paper aim to contribute to a deeper understanding of the complex landscape of AI reliability, providing a framework for researchers, practitioners, and policymakers to develop safer, more reliable AI systems that can be trusted to operate in environments where safety is paramount.

Using ultrasound shear wave elastography to characterize peripheral nerve mechanics: a systematic review on the normative reference values in healthy individuals.

Ultrasound shear wave elastography (SWE) is an emerging non-invasive imaging technique for peripheral nerve evaluation. Shear wave velocity (SWV), a surrogate measure of stiffness, holds promise as a biomarker for various peripheral nerve disorders. However, to maximize its clinical and biomechanical value, it is important to fully understand the factors that influence nerve SWV measurements. This systematic review aimed to identify the normal range of SWV for healthy sciatic and tibial nerves and to reveal the factors potentially affecting nerve SWV. An electronic search yielded 17 studies eligible for inclusion, involving 548 healthy individuals (age range, 17 to 72 years). Despite very good reliability metrics, the reported SWV values differed considerably across studies for the sciatic (1.9-9.9 m/s) and tibial (2.3-9.1 m/s) nerves. Factors such as measurement proximity to joint regions, limb postures inducing nerve axial stretching, and transducer alignment with nerve fiber orientation were associated with increased SWV. These findings suggest regional-specific nerve mechanical properties, non-linear elastic behaviour, and marked mechanical anisotropy. The impact of age and sex remains unclear and warrants further investigation. These results emphasize the importance of considering these factors when assessing and interpreting nerve SWE. While increased SWV has been linked to pathological changes affecting nerve tissue mechanics, the significant variability observed in healthy nerves highlights the need for standardized SWE assessment protocols. Developing guidelines for enhanced clinical utility and achieving a comprehensive understanding of the factors that influence nerve SWE assessments are critical in advancing the field.

LiFi-enabled UAV swarm networks.

Unmanned aerial vehicle (UAV) swarm communication is a powerful component of aerial relays; however, conventional radio frequency (RF)-based UAV swarm networks struggle to ensure timely and reliable communication. To this end, a light spectrum-based wireless system, LiFi, is presented to supplement in this work thanks to its distinctive benefits. We present the analytical derivation of the average block error probability (ABEP) as Chebyshev approximation, lower and upper bounds. Then, the key performance metrics of reliability, throughput, and latency expressions are provided as a function of the ABEP. The results show that the severe requirements of ultra-reliable (99.99%) and low-latency (sub-millisecond) communication (URLLC) are satisfied at even low signal-to-noise ratio (SNR) values. Additionally, in the numerical results, the impact of the blocklength, packet size, different distances among UAVs, SNR value, and light-emitting diode (LED) semi-angle are explored.

Effectiveness of Radiation Transport Variance Reduction Methods for Wide-Area Environmental Contamination Assay Applications

This study compares the accuracy, efficiency, and reliability of variance reduction (VR) methods for Monte Carlo radiation transport simulations involving wide-area ground plane (i.e., “surface”) and buried (i.e., “volumetric”) gamma source emissions from environmental soil. The simulation models are idealized external exposure scenarios intended as a basis for deriving site-specific dose-based or carcinogenic risk–based regulatory limits in the radiological site remediation process. These simulations are computationally resource intensive since particle tracks are transported from an extremely large source region to a relatively small detector region. For each simulation, several VR methods are compared with metrics of accuracy, efficiency, and reliability. The MCNP deterministic transport (DXTRAN) VR method was most effective for problems involving sources emitting low-energy gamma rays, and a coupled multicode method was more effective for problems involving sources emitting higher-energy gamma rays that undergo significant attenuation in the soil.

Reliability and Mean Time to System Failure of a Parallel System' by Using One or Two Decimal Random Data Points

Focusing on Weibull failure rules, which govern the stopping of components, this work evaluates reliability metrics such as stability and the mean time to system failure (MTSF) of a structure that is parallel. These metrics' behaviour has been seen for one or two decimal random values of component failure rates, operation times, form parameters, and the total quantity of components used in the parallel structure. In order to analyze the variation in the ethics of reliability as well as MTSF, the particular case of the Weibull distribution has also been taken up.

Efficiency, Precision, Validity, and Reliability of GlauCAT-Asian Computerized Adaptive Tests in Measuring Glaucoma-Related Quality of Life.

To determine the efficiency, precision, and agreement of GlauCAT-Asian and its corresponding validity and reliability. In this cross-sectional study, 219 participants (mean ± standard deviation age, 66.59 ± 8.61years; 34% female) across the spectrum of glaucoma severity and 50 glaucoma suspects were recruited from glaucoma clinics in Singapore. Participants answered seven computerized adaptive testing (CAT) evaluations (Ocular Comfort, Activity Limitation, Lighting, Mobility, Concerns, Psychosocial, Glaucoma Management) and underwent eye examinations. Efficiency (mean number of items required for each CAT and time taken for CAT versus full item banks [IBs]), agreement (concordance between CATs and full IB person measures, henceforth referred to as scores), and precision (standard error of measurement [SE]) were evaluated. Other validity and reliability metrics were also assessed. The mean number of items administered ranged from 9 (Mobility/Glaucoma Management) to 12 (Ocular Comfort). Compared to answering the full IBs, CATs provided an average time saving of 38.3% (range, 10% to 70.6% for Lighting and Activity Limitation, respectively). Agreement between scores obtained by CAT versus full IB was high (intracorrelation coefficient ≥0.75), as was precision of score estimates (mean SE range: 0.35 for Psychosocial to 0.29 for Mobility). Scores from Activity Limitation, Mobility, Lighting, and Concerns decreased significantly as glaucoma severity increased (criterion validity; P-trend <0.05). All tests displayed good convergent/divergent validity and test-retest reliability. GlauCAT-Asian provides efficient, precise, accurate, valid, and reliable measurement of the patient-centered impact of glaucoma. GlauCAT-Asian may provide a valuable clinical tool for ophthalmologists to monitor impact of disease progression and the effectiveness of therapies.

Business Intelligence: A Quantitative Approach

Business Intelligence has become in the front of demanding technologies. Many companies need technology to adapt to changing requirements, to deal with rich amounts of data; to get correct information, analysis data, and finally to quickly support decision-makers. Developing an agile BI application can be defined as a short-delivering software application. This process is based on frequently adapting to changing requirements, involving customer participation, and delivering high-quality applications. Agile teams need a set of reliable metrics to measure their performance at different three levels: functionality, content, and scalability. This paper proposes Goal Question Agility Metrics that can be used in measuring the performance of agile teams working in developing business intelligence applications. GQAM is based on the Goal Question Metrics method that was developed by Victor Basili in 1994 [19]. In addition, GQAM depends on agile and BI concepts. GQAM proposes a set of metrics that are used to measure performance. The GQAM will be subjected to experiments using a real data set in the future.

The factors influencing stress among nursing home staff: A cross-sectional study to develop stress measurement scale

This study aims to identify factors influencing job stress among nursing home staff in Malaysia. Understanding the job stress among nursing home staff is crucial to improving the quality of health care provided by staff in nursing homes. Despite studies of sources and levels of stress among nursing home professionals, effective ways to eliminate stressors among the nursing home staff in Malaysia are still under research. The role of social support as a mediator in the relationship between time pressure, role conflict, workload, job autonomy, and job stress in a nursing home was also examined to extend the underpinning theory. This study used a cross-sectional survey to collect data from the respondents to meet the research objective. The Smart Partial Least Squares (PLS) software tests the research model's hypotheses. The probability technique using cluster random sampling was employed to gather data for this study, which included 150 nursing home staff in Malaysia. The reliability and validity metrics were first assessed for the scale development process to guarantee internal consistency. The result revealed that the reliability and validity of each item used were established for the Nursing Home Staff Stress Measurement Scale. This study has indicated that time pressure and role conflict negatively influence social support. Job autonomy positively influenced job stress, while social support negatively influenced job stress.

Modeling Flexible Protein Structure With AlphaFold2 and Crosslinking Mass Spectrometry

We propose a pipeline that combines AlphaFold2 (AF2) and crosslinking mass spectrometry (XL-MS) to model the structure of proteins with multiple conformations. The pipeline consists of two main steps: ensemble generation using AF2 and conformer selection using XL-MS data. For conformer selection, we developed two scores-the monolink probability score (MP) and the crosslink probability score (XLP)-both of which are based on residue depth from the protein surface. We benchmarked MP and XLP on a large dataset of decoy protein structures and showed that our scores outperform previously developed scores. We then tested our methodology on three proteins having an open and closed conformation in the Protein Data Bank: Complement component 3 (C3), luciferase, and glutamine-binding periplasmic protein, first generating ensembles using AF2, which were then screened for the open and closed conformations using experimental XL-MS data. In five out of six cases, the most accurate model within the AF2 ensembles-or a conformation within 1Å of this model-was identified using crosslinks, as assessed through the XLP score. In the remaining case, only the monolinks (assessed through the MP score) successfully identified the open conformation of glutamine-binding periplasmic protein, and these results were further improved by including the "occupancy" of the monolinks. This serves as a compelling proof-of-concept for the effectiveness of monolinks. In contrast, the AF2 assessment score was only able to identify the most accurate conformation in two out of six cases. Our results highlight the complementarity of AF2 with experimental methods like XL-MS, with the MP and XLP scores providing reliable metrics to assess the quality of the predicted models. The MP and XLP scoring functions mentioned above are available at https://gitlab.com/topf-lab/xlms-tools.

Research on environmental reliability test and assessment of AI devices under vibration stress1

The proliferation of artificial intelligence (AI) devices has generated an increasing demand for reliability in their utilization. Nevertheless, the significant concern persists regarding the absence of suitable assessment and testing techniques to evaluate the performance of these intelligent systems in real-world conditions. In response to these issues, this paper conducts research on the reliability testing and assessment of AI visual perception systems under vibration stress. The paper introduces the working mechanism of the visual perception system and the various testing methods for AI devices. Based on this, a reliability assessment method for intelligent devices is proposed, which uses the Fréchet distance as the measurement function and environmental adaptability as the reliability metric. Additionally, a vibration test platform for the visual perception system is established, which offers a cost-effective and reliable solution to the high cost issue of field testing for AI devices. Finally, the reliability level of the visual perception system under various vibration conditions is tested through vibration testing. The research findings indicate that the reliability of AI models decreases as the degradation caused by vibration increases, following a normal distribution.

Protein homeostasis imprinting across evolution.

Protein homeostasis (a.k.a. proteostasis) is associated with the primary functions of life, and therefore with evolution. However, it is unclear how cellular proteostasis machines have evolved to adjust protein biogenesis needs to environmental constraints. Herein, we describe a novel computational approach, based on semantic network analysis, to evaluate proteostasis plasticity during evolution. We show that the molecular components of the proteostasis network (PN) are reliable metrics to deconvolute the life forms into Archaea, Bacteria and Eukarya and to assess the evolution rates among species. Semantic graphs were used as new criteria to evaluate PN complexity in 93 Eukarya, 250 Bacteria and 62 Archaea, thus representing a novel strategy for taxonomic classification, which provided information about species divergence. Kingdom-specific PN components were identified, suggesting that PN complexity may correlate with evolution. We found that the gains that occurred throughout PN evolution revealed a dichotomy within both the PN conserved modules and within kingdom-specific modules. Additionally, many of these components contribute to the evolutionary imprinting of other conserved mechanisms. Finally, the current study suggests a new way to exploit the genomic annotation of biomedical ontologies, deriving new knowledge from the semantic comparison of different biological systems.

Reliability-aware multi-objective approach for predictive asset management: A Danish distribution grid case study

The transition towards sustainable practices and a reliable electricity grid accommodates the rising electrification of the heating and transportation sectors. Aging, environmental factors, and operational conditions of electrical grid infrastructure contribute to a higher likelihood of faults. This leads to a reduced level of reliability, emphasizing the importance of renewing electrical grid infrastructure, particularly underground cables. Optimally replacing cables is essential, taking into account various factors like reducing the fault probability, minimizing the cost of power outages, and enhancing reliability within the budgetary constraint. This paper introduces an innovative methodology to predictive asset management for replacing underground cables using multi-objective optimization approach. Three objective functions are formulated: number of replaced cables, cost of power outages, and interruption-related index, which is determined through metrics like SAIFI, SAIDI, and ASIDI. These objectives are modeled as mixed-integer programming creating a multi-objective optimization problem, which is addressed using the epsilon-constraint approach. The optimization model identifies the cables that should be replaced within the budget constraint, aiming to optimize the objectives. The effectiveness of this approach is assessed using a real Danish distribution grid. The findings indicate that, compared to methods based on the cable age, fault vulnerability, and risk assessment, the proposed method demonstrates superior performance in terms of reliability metrics and power outage cost.

Study of reliability indicators of cable lines in rural areas

In this article, the investigation of the power supply system's reliability in our Republic, particularly focusing on the cable lines utilized in rural power networks, is highlighted as a pressing concern within the energy sector today. A crucial task in addressing this issue successfully involves researching the reliability metrics of existing cable lines, specifically self-contained insulated conductors in rural power networks with dynamic variable loads, and formulating pertinent recommendations. The reliability indicators of cable lines in rural areas were assessed using a comprehensive method that integrates modern approaches and the latest advancements in science and technology. The method's superiority over others is grounded in scientific principles. Furthermore, factors influencing the reliability metrics of cable lines in rural power networks, such as cable line failures, are systematically categorized.

Use and Failure Cause Analysis of Gas-Turbine Units at Distributed Generation Facilities

Distributed generation (DG) facilities commonly use gas-turbine units (GTU) varying in design and power. High manufacturing quality, reliability, efficiency, and low environmental impact are among the factors driving their popularity. This paper analyzes GTU reliability metrics and overviews the factors that may affect reliability of the units in use. It discusses the GTU design considerations that shall be borne in mind while engineering DG facilities and emergency automation devices. The paper further analyzes the recorded use GTUs at existing DG facilities from the perspective of the main causes and consequences of their failure.

Software Quality Assessment Approach using Analytical Hierarchical Model: Applied in SMEs

Continuous software quality evolution is crucial albeit challenging due to tight budgets and timelines. In fact, neglecting software quality in favour of customer satisfaction; either by reducing internal or external defects or by adding technical features shall place much pressure on project managers and stakeholders to find a balance between project budget and schedule. The increasing demand for high-quality software among stakeholders emphasizes the need for comprehensive approaches that can evaluate and rank quality metrics to maximize their benefits. In this research paper, we introduce an assessment approach developed using key quality metrics integrated from Cost Of Software Quality (COSQ) metrics, reliability metrics, and cost of defects metrics based on quantitative multicriteria decision analysis capabilities of the Analytic Hierarchy Process (AHP) model applied with a real case study for the software development team in Small and Medium Enterprises (SMEs) since most of them do not follow standards nor adopt well-structured quality methods. The evaluation is established based on the integrated AHP assessment approach as a practical solution for organizations seeking a critical examination of the relative significance of software metrics in terms of their utility in enhancing business performance. As a result, an evaluation of these quality metrics has been conducted to highlight the percentage of weight for each quality metric based on interviews with subject matters experts. The ranking of software quality metrics greatly helps stakeholders in selecting the most appropriate attribute for evaluating the developed software to identify high-impact quality initiatives and measure their effectiveness in the Software Development Life Cycle (SDLC)

The Effectiveness of Hidden Dependence Metrics in Bug Prediction

The Effectiveness of Hidden Dependence Metrics in Bug Prediction