Reliable Performance Assessment Research Articles

UNC Charlotte Autonomous Shuttle Pilot Study: An Assessment of Operational Performance, Reliability, and Challenges

This paper presents the findings from an autonomous shuttle pilot program conducted at the University of North Carolina at Charlotte between June and December 2023 as part of the North Carolina Department of Transportation’s Connected Autonomous Shuttle Supporting Innovation (CASSI) initiative. The shuttle completed 825 trips, transporting 565 passengers along a 2.2-mile mixed-traffic campus route. The study evaluates the shuttle’s operational performance, reliability, and challenges using data from onboard sensors, system logs, and operator reports. Key analyses are divided into four areas: service reliability, which assesses autonomy disengagements caused by signal loss, technical issues, and environmental factors; service robustness, focusing on the shuttle’s ability to maintain operations under adverse conditions; performance metrics, including average speed, autonomy percentage, and battery usage; and service usage, which examines the number of trips and passengers to gauge efficiency. Signal loss and battery-related issues were the primary causes of service interruptions, while environmental factors like weather and vegetation also affected shuttle performance. Recommendations include enhancing vehicle-to-infrastructure communication and optimizing battery management.

Urban biodiversity performance determining model (UrBioPDeM): The case of Isparta, Türkiye

Identifying, protecting, and developing biodiversity in urban environments contributes to ecosystem integrity. Although there are studies addressing biodiversity in natural areas in Türkiye, there has been no study specifically measuring the biodiversity of city centers. Adapting existing methods for measuring city biodiversity to Turkish cities is challenging due to their unique local dynamics. Therefore, there is a need to identify indicators that can be used to monitor and evaluate biodiversity performance for the protection and sustainability of urban areas in Türkiye and to develop a feasible, reliable, and measurable biodiversity performance assessment approach. The developed UrBioPDeM has 4 main factors and 32 sub-factors. Indicators for the assessment and management of urban biodiversity performance in Isparta, Türkiye, were identified and analyzed, and it was found that Isparta’s urban biodiversity has low potential. Therefore, suggestions were made to increase the urban biodiversity of Isparta. The developed approach is an appropriate and effective biodiversity performance model that can be applied in Turkish cities. The factors and sub-factors used in this method can be adjusted and revised according to the potential biodiversity characteristics of a city.

Junction Temperature Prediction and Lifetime Assessment in a PV Inverter Using a 10-year Mission profile

The expansion of great-scale photovoltaic (PV) power plants indicates the need for an accurate lifetime assessment of inverters to maintain energy supply availability. In this context, the study contributes in two ways. First, we use machine-learning (ML) models for junction temperature prediction. Second, we perform reliability assessments using a 10-year mission profile in three Brazilian cities. The thermal loadings are obtained through a look-up table approach. Although the ML models exhibit different performances in regression, other factors must be considered, such as easy-to-apply, interpretability, and generalization capability. The reliability assessment is typically based on an annual mission profile, assuming damage repeats until failure. However, only the historical series can confirm whether this choice was acceptable, pessimistic, or optimistic. For instance, in Campos do Jordão-SP, if the chosen mission profile is 2014, the expected failure of 10\% of inverter samples occurs three years earlier than suggested by the historical series. Regardless of the methodology used to estimate thermal loading or accumulated damage, the mission profile significantly influences photovoltaic inverter reliability, indicating that if more data is available, the chosen mission profile should align with the historical series.

Operational reliability and non-deterministic resilience estimation of active distribution network incorporating effect of real-time dynamic hosting capacity

Active distribution networks are increasingly recognized essential for achieving sustainable development goals. Traditionally, hosting capacity was considered as a static measure for planning distributed energy resources integration. This work introduces the concept of dynamic hosting capacity, which recurrently re-evaluates hosting capacity in response to erratic modern grid conditions. The introduction of dynamic hosting capacity facilitated testing variations of power injection from minimum to 100 %, sustaining power system governing parameter limits. This embarked the need of operational reliability assessment and enhancing situational awareness for optimum power injection and balance. To achieve operational reliability analysis based on dynamic hosting capacity, hybrid probability distribution function-based Monte Carlo simulation is proposed. This resulted in 85–90 %. improvisation of solar photovoltaic generation and load alignment, as this methodology provides comprehensive and accurate assessment of system performance under diverse uncertainties. The framework's validation includes projection of time-varying operational reliability indices, over time independent reliability indices i.e., dynamic loss of load probability, dynamic loss of load expectation, dynamic loss of load duration, dynamic loss of load frequency, dynamic grid margin, and dynamic grid dependency. This resulted in 30 % improvement in assessment of grid margin, facilitating reliable uncertainty handling competence. Additionally, expectation maximization algorithm is proposed to evaluate non-deterministic resilience due to ambiguities associated with solar photovoltaic distributed energy resources. The non-deterministic resilience assessment testified 80 % bounce-back rate, demonstrating better adaptability and robustness. The entire analysis is conducted in MATLAB, validated using Typhoon Hardware-in-Loop real-time platform, and compared with existing literatures to demonstrate its effectiveness.

Multi-Fidelity Aeromechanical Design Framework for High Flow Speed Multistage Axial Compressors

Abstract The development of novel engine architectures is vital in achieving the aviation sector’s net-zero carbon emission target by 2050. With today’s digital decade providing support for an accelerated technology maturation, the challenge for turbomachinery design remains to significantly push the limits of current performance within an ambitious development lead time. In this context, it is essential to adopt a design framework where the predictive models or simulations employed target a sufficiently reliable performance assessment. These models must be tailored to the dynamics of evolving industrial design processes, and therefore continuously balance required design flexibility, robust evaluation, appropriate fidelity (i.e. the level of detail and accuracy they provide), and resulting evaluation time. This paper discusses a framework for designing axial compressors and its application to the aeromechanical optimisation of a high-speed compressor rotor. The design environment integrates geometry parametrisation, modular evaluation with different levels of fidelity for the aerodynamic and structural models, and surrogate-based optimisation (SBO) capabilities. It is shown how the combination of a modular sequencing of the different models and the acceleration enabled by high-performance computing (HPC) and machine learning allows for a more advanced preliminary design. A significant gain in isentropic efficiency is attained while satisfying all structural constraints. At the same time, it is demonstrated that the framework is compatible with the characteristics of the preliminary design phase: both in its ability to adapt to cycle and design changes as well as regarding the turnaround time of the optimisation itself.

Operational reliability assessment of complex mechanical systems with multiple failure modes: An adaptive decomposition-synchronous-coordination approach

To effectively perform Complex Mechanical Systems Operational Reliability Assessment (CSORA) with multiple failures, the Adaptive Decomposition-Synchronous-Coordination Approach (ADSCA) is proposed by integrating Decomposition-Synchronous-Coordination strategy, adaptive modeling technology, surrogate model and multi-objective reliability assessment theory. In which, the main problem is decomposed into related sub-problems using a Decomposition-Synchronization-Coordination strategy, with sub-models coordinated to achieve synchronous mapping between multiple variables, thereby enhancing efficiency in large multi-subsystem problems. The adaptive modeling technique is adopted to construct a series of interrelated sub models based on the surrogate model as the basis function, improving performance. A multi-objective reliability assessment theory is adopted to achieve CSORA under multiple fault modes, where multiple weak links are integrated to comprehensively evaluate the reliability of complex systems. The significance of the ADSCA lies in its ability to manage complexity, enhance scalability, and improve the accuracy and efficiency of reliability assessments in complex mechanical systems. To demonstrate the effectiveness of the proposed approach, two illustrative examples are used. This includes approximation and probability analysis of nonlinear functions that exhibit multiple responses, as well as reliability analysis of temperature in civil aircraft braking system. The study can provide theoretical reference for the reliability evaluation of multi-failure operation in complex mechanical systems.

A Semi-Markov Stochastic Model for Operational Reliability Assessment of Hybrid AC and LCC-VSC-Based DC System With Remote Wind Farms

A Semi-Markov Stochastic Model for Operational Reliability Assessment of Hybrid AC and LCC-VSC-Based DC System With Remote Wind Farms

Training framework for high-stakes OSCE: Experience from volunteer standardized patients' bank.

Training of standardized patients (SPs) for national high-stakes OSCE helps to ensure a reliable assessment of student performance in various clinical scenarios. However, workflow protocols to train SPs vary. Medical schools adopt specific measures to ensure standardization. We present a development workflow of the SPs' training framework for high-stakes OSCE using a volunteer SPs' bank. Our approach was guided by the social learning theory. Three educators and 17/20 (85%) members of our volunteer SPs' bank worked in a collaborative partnership on the construction of pedagogical content of the training framework comprising three 2-hour sessions. Since SPs have to demonstrate acquired behaviors, intended learning outcomes used the words "apply", "perform" and "participate." A principal part of the evaluation was the achievement of intended learning outcomes by the SPs during 3 formative OSCEs. Seventeen SPs, 356 fourth year medical students and 60 examiners participated. Quantitative and qualitative data were collected by post-session questionnaires and analyzed using descriptive statistics and thematic analysis. Twelve examiners evaluated a mean of 29.7+/-0.14 SD patient-student encounters. In total, 15/16 SPs (93.8%) considered the contact with students as easy and 4 SPs (31%) reported the experience as stressful. Two themes emerged from the free-text comments: "Gaining experience as SP" and "Concerns for evaluated students." The proposed SPs' training framework for high-stakes OSCE may be useful for other medical disciplines and health professions initiating SP-based assessment programs. The strategy of development and evaluation are outlined to guide a successful application of the curriculum standards.

Use of optical see-through head-mounted devices in dentistry - a scoping review.

The aim of this scoping review was to identify the scientific evidence related to the utilization of Optical See- Through Head-Mounted Display (OST-HMD) in dentistry, and to determine future research needs. The research question was formulated using the "Population" (P), "Concept" (Cpt), and "Context" (Cxt) framework for scoping reviews. Existing literature was designated as P, OST-HMD as Cpt, and Dentistry as Cxt. An electronic search was conducted in PubMed, Embase, Web of Science, and CENTRAL. Two authors independently screened titles and abstracts and performed the full-text analysis. The search identified 286 titles after removing duplicates. Nine studies, involving 138 participants and 1760 performed tests were included in this scoping review. Seven of the articles were preclinical studies, one was a survey, and one was a clinical trial. The included manuscripts covered various dental fields: three studies in orthodontics, two in oral surgery, two in conservative dentistry, one in general dentistry, and the remaining one in prosthodontics. Five articles focused on educational purposes. Two brands of OST-HMD were used: in eight studies HoloLens Microsoft was used, while Google Glass was utilized in one article. The overall number of included studies was low; therefore, the available data from this review cannot yet support an evidence-based recommendation for the clinical use of OST-HMDs. However, the existing preclinical data indicate a significant capacity for clinical and educational implementation. Further adoption of these devices will facilitate more reliable and objective quality and performance assessments, as well as more direct comparisons with conventional workflows. More clinical studies must be conducted to substantiate the potential benefits and reliability for patients and clinicians.

Proficiency testing for event-specific quantification of genetically modified maize MON87427: Performance assessment based on the metrologically traceable reference values as assigned values

The Asia Pacific Metrology Program and the Accreditation Cooperation joint Proficiency Testing (PT) program for the quantification of genetically modified maize MON87427 was organized by the National Institute of Metrology, China, to enhance the measurement accuracy and metrological traceability in the region. Certified reference materials were employed as test samples; metrologically traceable certified reference values served as PT reference values (PTRVs) for evaluating the participants results. The consensus values obtained from the participants were higher than the assigned values, potentially due to the systematic effects of DNA extraction process. The participants' relatively poor overall performance by the ζ-score compared with z-score demonstrates their need to thoroughly investigate quantification bias to elevate the measurement capability of genetically modified (GM) content and deepen their understanding of uncertainty estimation. This program confirmed the importance of using metrologically traceable reference values instead of consensus values as PTRV for reliable performance assessment.

Machine Learning Library in C++

This project aims to develop a comprehensive and versatile machine learning library in C++ tailored to address the diverse needs of developers and researchers in the field. The library encompasses a robust set of core machine learning algorithms, encompassing supervised, unsupervised, and reinforcement learning techniques. Additionally, it incorporates essential data preprocessing tools to streamline data manipulation and feature engineering tasks, along with model evaluation capabilities crucial for assessing algorithm performance. The library's primary focus is on providing a rich suite of machine learning algorithms, This empowers users to effectively prepare data for training machine learning models. Additionally, the library provides tools for data splitting, and model evaluation to ensure reliable and robust model performance assessment.

Operational Reliability Analysis of Turbine Blisk Using an Enhanced Moving Neural Network Framework

As one of the key components of an aeroengine, turbine blisk endures complex coupling loads under a harsh operational environment so that the reliability of turbine blisk directly influences the safe operation of aeroengine. It is urgent to precisely perform the reliability estimation of a complex blisk structure. To address this issue, an enhanced Moving Neural Network Framework (MNNF) is proposed by integrating compact support region theory, improve sooty tern optimization algorithm (ISTOA), and Bayesian regularization strategy into artificial neural network. The compact support region theory is applied to select the efficient samples for modeling from the training samples set, the ISTOA is to determine the optimal compact support region, and Bayesian regularization thought is utilized to improve the generalization ability of neural network model. The operational reliability assessment of aeroengine blisk is performed with the consideration of transient loads to verify the proposed MNNF method. It is shown that the reliability degree of turbine blisk stain is 0.9984 when the allowable value is 5.2862 × 10−3 m. In line with the comparison of methods, the developed MNNF approach has 0.99738 in root means square error, 3.1634 × 10−4 m in goodness of fit, 0.423 s in modeling time, 99.99% in simulation precision, and 0.496 s in simulation time under 10,000 simulations, which are superior to all other methods (i.e., 99.96%, 99.91%, 99.93%, 99.97%, and 99.97% in simulation precision and 16.27%, 4.82%, 30.07%, 39.87%, and 23.59% in simulation efficiency, for the response surface method (RSM), Kriging, support vector machine (SVM), back propagation-artificial neural network (BP-NN), and BP-NN based on particle swarm optimization (BP-PSO) methods, respectively). It is demonstrated that the MNNF method holds excellent modeling and simulation performances. The efforts of this study provide promising tools and insights into the reliability design of complex structures, and enrich and develop reliability theory.

An Analytical Reliability Performance Assessment of Cell-Free Massive MIMO Systems for 5G and Beyond Cellular Networks

In today‟s realm, the 5G cellular networks offer a new era of fully digital environment by connecting people anywhere anytime. This era requires inspiring demands to the networks in terms of spectral and energy efficiencies, low-latency and ultra-reliable communications. A major development to deal with these unparalleled demands is the introduction of Cell- Free Massive MIMO (CF mMIMO) systems to the cellular network. In recent years, mathematical foundations have been laid for a CF mMIMO communication system to evaluate the spectral and energy efficiencies given the number of access points. Various existing literatures have adopted mathematical models to formulate closed-form expressions for these efficiencies using uncorrelated Rayleigh fading for simple linear processing detectors. In this paper, we try to address a compelling and interesting question which has not received much attention lately. The question is how to evaluate the reliability of a connected user terminal in both a CF mMIMO and conventional mMIMO system? We answer this question by first analyzing the signal models for both systems assuming exact channel estimates and consider an uncorrelated Rayleigh fading channel model. The exact Bit Error Rate, as a reliability assessment, in closed-form are then derived based on strong mathematical foundations, which are simple to numerically evaluate in Matlab R2022, for MPSK and MQAM modulation schemes. The mathematical models presented in this work can be used in 5G and Beyond 5G networks to evaluate the distributions of error rates for a user given the number of Access points or Base Station antennas and modulation scheme. Machine Learning algorithms can be applied to the analytical expressions with a view to predict the error performance of a user terminal given the fading characteristic of the propagation environmen

A Review of Cross-Scale Theoretical Contact Models for Bolted Joints Interfaces

Bolted joints structures are critical fastening components widely used in mechanical equipment. Under long-term loading conditions, the bolted joints interface generates strong nonlinearities within the system. The nonlinear stiffness inside the bolt leads to changes in the stiffness of the whole system. This affects the dynamic characteristics of the whole system. It brings challenges and difficulties to the performance prediction and reliability assessment of the equipment. A cross-scale theoretical model study based on the microscopic contact mechanism can provide a more comprehensive understanding and cognition of the degradation behavior of bolted joints interfaces. The current development status and deformation process of asperity models are summarized. The research progress of statistical summation model and contact fractal model based on microscopic contact mechanism is analyzed. The experimental methods for parameter identification of connection interfaces are reviewed. The study of numerical modelling of bolted joints structures from the surface contact mechanism is briefly described. Future research directions for cross-scale modelling of bolted joints structures are outlined.

Electric current stressing enhanced damping properties in Sn5Sb solder

PurposeThe purpose of this paper is to investigate the effects of electric current stressing on damping properties of Sn5Sb solder.Design/methodology/approachUniformly shaped Sn5Sb solders were prepared as samples. The length, width and thickness of the samples were 60.0, 5.0 and 0.5 mm, respectively. The damping properties of the samples were tested by dynamic mechanical analyzer with a cooling system to control the test temperature in the range of −100 to 100°C. Simultaneously, electric current was imposed to the tested samples using a direct current supply. After tests, the samples were characterized using scanning electron microscope, electron backscatter diffraction and transmission electron microscope, which was aimed to figure out the damping mechanism in terms of electric current stressing induced microstructure evolution.FindingsIt is confirmed experimentally that the increase in damping properties is due to Joule heating and athermal effects of current stressing, in which Joule heating should make a higher contribution. G–L theory can be used to explain the damping properties of strain amplitude under current stressing by quantitative description of geometrically necessary dislocation density. While the critical strain amplitude and high temperature activation energy decrease with increasing electric current.Originality/valueThese results provide a new method for vibration reliability evaluation of high-temperature lead-free solders in serving electronics. Notably, this method should be also inspiring for the mechanical performance evaluation and reliability assessment of conductive materials and structures serving under electric current stressing.

Machine Learning-Based Framework for Failure Forecast and Shear Strength Estimation of Non-Conforming RC Shear Walls

ABSTRACT Earthquake reconnaissance after the recent Kahramanmaras, Turkey, earthquake sequence in February 6, 2023 has shown that the majority of the reinforced concrete (RC) structural elements do not comply with the Turkish Building Earthquake Code (TBEC 2018) requirements. Accurate estimation of the actual strength of existing non-compliant RC structural members is critical to accomplish reliable earthquake performance assessments. This study aims to propose a machine learning-based framework to establish a realistic failure forecast as well as an accurate shear strength estimation depending on the expected failure modes of such walls. To achieve this, shear wall design properties are assigned as inputs whereas failure mode and shear strength were designated as outputs for the corresponding classification and regression problems, respectively. Widely used machine learning methods, namely: Ridge Linear Regression, Logistic Regression, Multi-layer Perceptron, Support Vector Machine, Random Forest, XGBoost, LightGBM, and CatBoost are employed using a database consisting of 432 non-conforming shear walls. The data is randomly split to train and test sets (80% and 20%, respectively) to develop the predictive models and to evaluate model performances, respectively. Performance metrics (e.g. R 2 , RMSE) are evaluated over a hundred random splits to ensure robustness. The proposed CatBoost-based models achieve overall accuracies above 90% for both failure prediction and shear strength estimation. The proposed framework is believed to be a promising tool for earthquake engineers for practical and accurate failure mode forecast and shear strength estimation of non-compliant RC shear walls.

IBCA: An Intelligent Platform for Social Insurance Benefit Qualification Status Assessment

Social insurance benefits qualification assessment is an important task to ensure that retirees enjoy their benefits according to the regulations. It also plays a key role in curbing social security frauds. In this paper, we report the deployment of the Intelligent Benefit Certification and Analysis (IBCA) platform, an AI-empowered platform for verifying the status of retirees to ensure proper dispursement of funds in Shandong province, China. Based on an improved Gated Recurrent Unit (GRU) neural network, IBCA aggregates missing value interpolation, temporal information, and global and local feature extraction to perform accurate retiree survival rate prediction. Based on the predicted results, a reliability assessment mechanism based on Variational Auto-Encoder (VAE) and Monte-Carlo Dropout (MC Dropout) is executed to perform reliability assessment. Deployed since November 2019, the IBCA platform has been adopted by 12 cities across the Shandong province, handling over 50 terabytes of data. It has empowered human resources and social services, civil affairs, and health care institutions to collaboratively provide high-quality public services. Under the IBCA platform, the efficiency of resources utilization as well as the accuracy of benefit qualification assessment have been significantly improved. It has helped Dareway Software Co. Ltd earn over RMB 50 million of revenue.

Analyzing human reliability for the operation of cargo oil pump using fuzzy CREAM extended Bayesian Network (BN)

Crude oil cargo discharging operations can be performed by cargo oil pumps in tanker ships. Considering the harmful effects of crude oil, any failure during the cargo oil pump operation may pose acute hazards such as temperature and pressure increase in pipelines, leakage, fire, and following undesired events that lead to marine environmental pollution and toxic effects to humans. The process is managed by the ship's crew, and it requires following the operational steps in order. This paper aims at performing a detailed human reliability analysis (HRA) for the operational process of cargo oil pumps on a tanker ship. In the paper, the Cognitive Reliability and Error Analysis Method (CREAM) is used to calculate human error probability under a fuzzy set which deals with the uncertainty and ambiguity of the CPC (Common Performance Condition). The Bayesian Network (BN) approach is adopted to determine the probability distribution of the control modes in reliability assessment. In view of the findings, the human reliability for the entire process of cargo oil pump operation on tanker ship is found 8.82E-01. The outcomes of the paper provide valuable insight into the improvement of tanker safety in shipboard operation and performance reliability assessment.

Performance Evaluation of Hospitals under Data Uncertainty: An Uncertain Common-Weights Data Envelopment Analysis.

In the context of healthcare systems, the performance evaluation of hospitals plays a crucial role in assessing the quality of healthcare systems and facilitating informed decision-making processes. However, the presence of data uncertainty poses significant challenges to accurate performance measurement. This paper presents a novel uncertain common-weights data envelopment analysis (UCWDEA) approach for evaluating the performance of hospitals under uncertain environments. The proposed UCWDEA approach addresses the limitations of traditional data envelopment analysis (DEA) models by incorporating the uncertainty theory (UT) to model the inherent uncertainty in input and output data. Also, by utilizing a common set of weights (CSW) technique, the UCWDEA method provides a more robust and reliable assessment of hospital performance. The main advantages of the proposed UCWDEA approach can be succinctly summarized as follows. Firstly, it allows for the comparison of all hospitals on a consistent basis to calculate a realistic efficiency score, rather than an overly optimistic efficiency score. Secondly, the uncertain common-weights DEA approach exhibits linearity, enhancing its applicability. Thirdly, it possesses the capability to extend its utility under various other prevalent uncertainty distributions. Moreover, it enhances the discriminatory power of results, facilitates the ranking of hospitals in the presence of data uncertainty, and aids in identifying the sensitivity and stability levels of hospitals towards data uncertainty. Notably, in order to showcase the pragmatic application and efficacy of the uncertain common-weights DEA model, a genuine dataset has been utilized to evaluate the efficiency of 20 public hospitals in Tehran, all of which are affiliated with the Iran University of Medical Sciences. The results of the experiment demonstrate the efficacy of the UCWDEA approach in assessing and ranking hospitals amidst uncertain conditions. In summary, the research outcomes can offer policymakers valuable insights regarding hospital performance amidst data uncertainty. Additionally, it can provide practical recommendations on optimizing resource allocation, benchmarking performance, and formulating effective policies to augment the overall efficiency and effectiveness of healthcare services.

Power system operational reliability assessment based on the data center energy consumption elastic space

In the era of big data, data centers with high energy consumption, interconnectivity, and load flexibility have developed rapidly. However, due to data privacy issues, the traditional power-system operational reliability assessment (ORA) struggles to precisely consider the load flexibility of data centers, leading to inaccurate evaluation. To this end, this article proposes an ORA method considering the load flexibility of data centers via the energy consumption elastic space. By transforming the inner operation constraints of data centers into an equivalent elastic space, the ORA does not require any private data to complete the evaluation. Specifically, the energy consumption model of data centers is established to accurately describe the load flexibility. Then, based on multi-parametric programming techniques, the energy consumption elastic space of data centers is characterized by data centers’ power demand constraints, which do not involve privacy data, and no privacy concerns exist. Finally, the ORA model and the evaluation method based on the energy consumption elastic space can be constructed. With a lot of data center operation constraints being replaced by power demand constraints, the proposed method can complete an evaluation faster without accuracy loss. Its effectiveness is validated through simulations using the IEEE RTS 24-bus system and a provincial 661-bus system.