Reliable Performance Assessment Research Articles

Training framework for high-stakes OSCE: Experience from volunteer standardized patients' bank.

Training of standardized patients (SPs) for national high-stakes OSCE helps to ensure a reliable assessment of student performance in various clinical scenarios. However, workflow protocols to train SPs vary. Medical schools adopt specific measures to ensure standardization. We present a development workflow of the SPs' training framework for high-stakes OSCE using a volunteer SPs' bank. Our approach was guided by the social learning theory. Three educators and 17/20 (85%) members of our volunteer SPs' bank worked in a collaborative partnership on the construction of pedagogical content of the training framework comprising three 2-hour sessions. Since SPs have to demonstrate acquired behaviors, intended learning outcomes used the words "apply", "perform" and "participate." A principal part of the evaluation was the achievement of intended learning outcomes by the SPs during 3 formative OSCEs. Seventeen SPs, 356 fourth year medical students and 60 examiners participated. Quantitative and qualitative data were collected by post-session questionnaires and analyzed using descriptive statistics and thematic analysis. Twelve examiners evaluated a mean of 29.7+/-0.14 SD patient-student encounters. In total, 15/16 SPs (93.8%) considered the contact with students as easy and 4 SPs (31%) reported the experience as stressful. Two themes emerged from the free-text comments: "Gaining experience as SP" and "Concerns for evaluated students." The proposed SPs' training framework for high-stakes OSCE may be useful for other medical disciplines and health professions initiating SP-based assessment programs. The strategy of development and evaluation are outlined to guide a successful application of the curriculum standards.

Use of optical see-through head-mounted devices in dentistry - a scoping review.

The aim of this scoping review was to identify the scientific evidence related to the utilization of Optical See- Through Head-Mounted Display (OST-HMD) in dentistry, and to determine future research needs. The research question was formulated using the "Population" (P), "Concept" (Cpt), and "Context" (Cxt) framework for scoping reviews. Existing literature was designated as P, OST-HMD as Cpt, and Dentistry as Cxt. An electronic search was conducted in PubMed, Embase, Web of Science, and CENTRAL. Two authors independently screened titles and abstracts and performed the full-text analysis. The search identified 286 titles after removing duplicates. Nine studies, involving 138 participants and 1760 performed tests were included in this scoping review. Seven of the articles were preclinical studies, one was a survey, and one was a clinical trial. The included manuscripts covered various dental fields: three studies in orthodontics, two in oral surgery, two in conservative dentistry, one in general dentistry, and the remaining one in prosthodontics. Five articles focused on educational purposes. Two brands of OST-HMD were used: in eight studies HoloLens Microsoft was used, while Google Glass was utilized in one article. The overall number of included studies was low; therefore, the available data from this review cannot yet support an evidence-based recommendation for the clinical use of OST-HMDs. However, the existing preclinical data indicate a significant capacity for clinical and educational implementation. Further adoption of these devices will facilitate more reliable and objective quality and performance assessments, as well as more direct comparisons with conventional workflows. More clinical studies must be conducted to substantiate the potential benefits and reliability for patients and clinicians.

Machine Learning Library in C++

This project aims to develop a comprehensive and versatile machine learning library in C++ tailored to address the diverse needs of developers and researchers in the field. The library encompasses a robust set of core machine learning algorithms, encompassing supervised, unsupervised, and reinforcement learning techniques. Additionally, it incorporates essential data preprocessing tools to streamline data manipulation and feature engineering tasks, along with model evaluation capabilities crucial for assessing algorithm performance. The library's primary focus is on providing a rich suite of machine learning algorithms, This empowers users to effectively prepare data for training machine learning models. Additionally, the library provides tools for data splitting, and model evaluation to ensure reliable and robust model performance assessment.

Operational Reliability Analysis of Turbine Blisk Using an Enhanced Moving Neural Network Framework

As one of the key components of an aeroengine, turbine blisk endures complex coupling loads under a harsh operational environment so that the reliability of turbine blisk directly influences the safe operation of aeroengine. It is urgent to precisely perform the reliability estimation of a complex blisk structure. To address this issue, an enhanced Moving Neural Network Framework (MNNF) is proposed by integrating compact support region theory, improve sooty tern optimization algorithm (ISTOA), and Bayesian regularization strategy into artificial neural network. The compact support region theory is applied to select the efficient samples for modeling from the training samples set, the ISTOA is to determine the optimal compact support region, and Bayesian regularization thought is utilized to improve the generalization ability of neural network model. The operational reliability assessment of aeroengine blisk is performed with the consideration of transient loads to verify the proposed MNNF method. It is shown that the reliability degree of turbine blisk stain is 0.9984 when the allowable value is 5.2862 × 10−3 m. In line with the comparison of methods, the developed MNNF approach has 0.99738 in root means square error, 3.1634 × 10−4 m in goodness of fit, 0.423 s in modeling time, 99.99% in simulation precision, and 0.496 s in simulation time under 10,000 simulations, which are superior to all other methods (i.e., 99.96%, 99.91%, 99.93%, 99.97%, and 99.97% in simulation precision and 16.27%, 4.82%, 30.07%, 39.87%, and 23.59% in simulation efficiency, for the response surface method (RSM), Kriging, support vector machine (SVM), back propagation-artificial neural network (BP-NN), and BP-NN based on particle swarm optimization (BP-PSO) methods, respectively). It is demonstrated that the MNNF method holds excellent modeling and simulation performances. The efforts of this study provide promising tools and insights into the reliability design of complex structures, and enrich and develop reliability theory.

An Analytical Reliability Performance Assessment of Cell-Free Massive MIMO Systems for 5G and Beyond Cellular Networks

In today‟s realm, the 5G cellular networks offer a new era of fully digital environment by connecting people anywhere anytime. This era requires inspiring demands to the networks in terms of spectral and energy efficiencies, low-latency and ultra-reliable communications. A major development to deal with these unparalleled demands is the introduction of Cell- Free Massive MIMO (CF mMIMO) systems to the cellular network. In recent years, mathematical foundations have been laid for a CF mMIMO communication system to evaluate the spectral and energy efficiencies given the number of access points. Various existing literatures have adopted mathematical models to formulate closed-form expressions for these efficiencies using uncorrelated Rayleigh fading for simple linear processing detectors. In this paper, we try to address a compelling and interesting question which has not received much attention lately. The question is how to evaluate the reliability of a connected user terminal in both a CF mMIMO and conventional mMIMO system? We answer this question by first analyzing the signal models for both systems assuming exact channel estimates and consider an uncorrelated Rayleigh fading channel model. The exact Bit Error Rate, as a reliability assessment, in closed-form are then derived based on strong mathematical foundations, which are simple to numerically evaluate in Matlab R2022, for MPSK and MQAM modulation schemes. The mathematical models presented in this work can be used in 5G and Beyond 5G networks to evaluate the distributions of error rates for a user given the number of Access points or Base Station antennas and modulation scheme. Machine Learning algorithms can be applied to the analytical expressions with a view to predict the error performance of a user terminal given the fading characteristic of the propagation environmen

Proficiency testing for event-specific quantification of genetically modified maize MON87427: Performance assessment based on the metrologically traceable reference values as assigned values

The Asia Pacific Metrology Program and the Accreditation Cooperation joint Proficiency Testing (PT) program for the quantification of genetically modified maize MON87427 was organized by the National Institute of Metrology, China, to enhance the measurement accuracy and metrological traceability in the region. Certified reference materials were employed as test samples; metrologically traceable certified reference values served as PT reference values (PTRVs) for evaluating the participants' results. The consensus values obtained from the participants were higher than the assigned values, potentially due to the systematic effects of DNA extraction process. The participants' relatively poor overall performance by the ζ-score compared with z-score demonstrates their need to thoroughly investigate quantification bias to elevate the measurement capability of genetically modified (GM) content and deepen their understanding of uncertainty estimation. This program confirmed the importance of using metrologically traceable reference values instead of consensus values as PTRV for reliable performance assessment.

A Review of Cross-Scale Theoretical Contact Models for Bolted Joints Interfaces

Bolted joints structures are critical fastening components widely used in mechanical equipment. Under long-term loading conditions, the bolted joints interface generates strong nonlinearities within the system. The nonlinear stiffness inside the bolt leads to changes in the stiffness of the whole system. This affects the dynamic characteristics of the whole system. It brings challenges and difficulties to the performance prediction and reliability assessment of the equipment. A cross-scale theoretical model study based on the microscopic contact mechanism can provide a more comprehensive understanding and cognition of the degradation behavior of bolted joints interfaces. The current development status and deformation process of asperity models are summarized. The research progress of statistical summation model and contact fractal model based on microscopic contact mechanism is analyzed. The experimental methods for parameter identification of connection interfaces are reviewed. The study of numerical modelling of bolted joints structures from the surface contact mechanism is briefly described. Future research directions for cross-scale modelling of bolted joints structures are outlined.

Electric current stressing enhanced damping properties in Sn5Sb solder

PurposeThe purpose of this paper is to investigate the effects of electric current stressing on damping properties of Sn5Sb solder.Design/methodology/approachUniformly shaped Sn5Sb solders were prepared as samples. The length, width and thickness of the samples were 60.0, 5.0 and 0.5 mm, respectively. The damping properties of the samples were tested by dynamic mechanical analyzer with a cooling system to control the test temperature in the range of −100 to 100°C. Simultaneously, electric current was imposed to the tested samples using a direct current supply. After tests, the samples were characterized using scanning electron microscope, electron backscatter diffraction and transmission electron microscope, which was aimed to figure out the damping mechanism in terms of electric current stressing induced microstructure evolution.FindingsIt is confirmed experimentally that the increase in damping properties is due to Joule heating and athermal effects of current stressing, in which Joule heating should make a higher contribution. G–L theory can be used to explain the damping properties of strain amplitude under current stressing by quantitative description of geometrically necessary dislocation density. While the critical strain amplitude and high temperature activation energy decrease with increasing electric current.Originality/valueThese results provide a new method for vibration reliability evaluation of high-temperature lead-free solders in serving electronics. Notably, this method should be also inspiring for the mechanical performance evaluation and reliability assessment of conductive materials and structures serving under electric current stressing.

Machine Learning-Based Framework for Failure Forecast and Shear Strength Estimation of Non-Conforming RC Shear Walls

ABSTRACT Earthquake reconnaissance after the recent Kahramanmaras, Turkey, earthquake sequence in February 6, 2023 has shown that the majority of the reinforced concrete (RC) structural elements do not comply with the Turkish Building Earthquake Code (TBEC 2018) requirements. Accurate estimation of the actual strength of existing non-compliant RC structural members is critical to accomplish reliable earthquake performance assessments. This study aims to propose a machine learning-based framework to establish a realistic failure forecast as well as an accurate shear strength estimation depending on the expected failure modes of such walls. To achieve this, shear wall design properties are assigned as inputs whereas failure mode and shear strength were designated as outputs for the corresponding classification and regression problems, respectively. Widely used machine learning methods, namely: Ridge Linear Regression, Logistic Regression, Multi-layer Perceptron, Support Vector Machine, Random Forest, XGBoost, LightGBM, and CatBoost are employed using a database consisting of 432 non-conforming shear walls. The data is randomly split to train and test sets (80% and 20%, respectively) to develop the predictive models and to evaluate model performances, respectively. Performance metrics (e.g. R 2 , RMSE) are evaluated over a hundred random splits to ensure robustness. The proposed CatBoost-based models achieve overall accuracies above 90% for both failure prediction and shear strength estimation. The proposed framework is believed to be a promising tool for earthquake engineers for practical and accurate failure mode forecast and shear strength estimation of non-compliant RC shear walls.

IBCA: An Intelligent Platform for Social Insurance Benefit Qualification Status Assessment

Social insurance benefits qualification assessment is an important task to ensure that retirees enjoy their benefits according to the regulations. It also plays a key role in curbing social security frauds. In this paper, we report the deployment of the Intelligent Benefit Certification and Analysis (IBCA) platform, an AI-empowered platform for verifying the status of retirees to ensure proper dispursement of funds in Shandong province, China. Based on an improved Gated Recurrent Unit (GRU) neural network, IBCA aggregates missing value interpolation, temporal information, and global and local feature extraction to perform accurate retiree survival rate prediction. Based on the predicted results, a reliability assessment mechanism based on Variational Auto-Encoder (VAE) and Monte-Carlo Dropout (MC Dropout) is executed to perform reliability assessment. Deployed since November 2019, the IBCA platform has been adopted by 12 cities across the Shandong province, handling over 50 terabytes of data. It has empowered human resources and social services, civil affairs, and health care institutions to collaboratively provide high-quality public services. Under the IBCA platform, the efficiency of resources utilization as well as the accuracy of benefit qualification assessment have been significantly improved. It has helped Dareway Software Co. Ltd earn over RMB 50 million of revenue.

Analyzing human reliability for the operation of cargo oil pump using fuzzy CREAM extended Bayesian Network (BN)

Crude oil cargo discharging operations can be performed by cargo oil pumps in tanker ships. Considering the harmful effects of crude oil, any failure during the cargo oil pump operation may pose acute hazards such as temperature and pressure increase in pipelines, leakage, fire, and following undesired events that lead to marine environmental pollution and toxic effects to humans. The process is managed by the ship's crew, and it requires following the operational steps in order. This paper aims at performing a detailed human reliability analysis (HRA) for the operational process of cargo oil pumps on a tanker ship. In the paper, the Cognitive Reliability and Error Analysis Method (CREAM) is used to calculate human error probability under a fuzzy set which deals with the uncertainty and ambiguity of the CPC (Common Performance Condition). The Bayesian Network (BN) approach is adopted to determine the probability distribution of the control modes in reliability assessment. In view of the findings, the human reliability for the entire process of cargo oil pump operation on tanker ship is found 8.82E-01. The outcomes of the paper provide valuable insight into the improvement of tanker safety in shipboard operation and performance reliability assessment.

Performance Evaluation of Hospitals under Data Uncertainty: An Uncertain Common-Weights Data Envelopment Analysis.

In the context of healthcare systems, the performance evaluation of hospitals plays a crucial role in assessing the quality of healthcare systems and facilitating informed decision-making processes. However, the presence of data uncertainty poses significant challenges to accurate performance measurement. This paper presents a novel uncertain common-weights data envelopment analysis (UCWDEA) approach for evaluating the performance of hospitals under uncertain environments. The proposed UCWDEA approach addresses the limitations of traditional data envelopment analysis (DEA) models by incorporating the uncertainty theory (UT) to model the inherent uncertainty in input and output data. Also, by utilizing a common set of weights (CSW) technique, the UCWDEA method provides a more robust and reliable assessment of hospital performance. The main advantages of the proposed UCWDEA approach can be succinctly summarized as follows. Firstly, it allows for the comparison of all hospitals on a consistent basis to calculate a realistic efficiency score, rather than an overly optimistic efficiency score. Secondly, the uncertain common-weights DEA approach exhibits linearity, enhancing its applicability. Thirdly, it possesses the capability to extend its utility under various other prevalent uncertainty distributions. Moreover, it enhances the discriminatory power of results, facilitates the ranking of hospitals in the presence of data uncertainty, and aids in identifying the sensitivity and stability levels of hospitals towards data uncertainty. Notably, in order to showcase the pragmatic application and efficacy of the uncertain common-weights DEA model, a genuine dataset has been utilized to evaluate the efficiency of 20 public hospitals in Tehran, all of which are affiliated with the Iran University of Medical Sciences. The results of the experiment demonstrate the efficacy of the UCWDEA approach in assessing and ranking hospitals amidst uncertain conditions. In summary, the research outcomes can offer policymakers valuable insights regarding hospital performance amidst data uncertainty. Additionally, it can provide practical recommendations on optimizing resource allocation, benchmarking performance, and formulating effective policies to augment the overall efficiency and effectiveness of healthcare services.

Power system operational reliability assessment based on the data center energy consumption elastic space

In the era of big data, data centers with high energy consumption, interconnectivity, and load flexibility have developed rapidly. However, due to data privacy issues, the traditional power-system operational reliability assessment (ORA) struggles to precisely consider the load flexibility of data centers, leading to inaccurate evaluation. To this end, this article proposes an ORA method considering the load flexibility of data centers via the energy consumption elastic space. By transforming the inner operation constraints of data centers into an equivalent elastic space, the ORA does not require any private data to complete the evaluation. Specifically, the energy consumption model of data centers is established to accurately describe the load flexibility. Then, based on multi-parametric programming techniques, the energy consumption elastic space of data centers is characterized by data centers’ power demand constraints, which do not involve privacy data, and no privacy concerns exist. Finally, the ORA model and the evaluation method based on the energy consumption elastic space can be constructed. With a lot of data center operation constraints being replaced by power demand constraints, the proposed method can complete an evaluation faster without accuracy loss. Its effectiveness is validated through simulations using the IEEE RTS 24-bus system and a provincial 661-bus system.

Accelerating the Assessment of Hysteresis in Perovskite Solar Cells.

Halide perovskite materials have reached important milestones in the photovoltaic field, positioning them as realistic alternatives to conventional solar cells. However, unavoidable kinetic phenomena have represented a major concern for reliable steady-state performance assessment from standard current-voltage measurements. In particular, the dynamic hysteresis of current-voltage curves requires relatively long stabilization to achieve a credible figure for the power conversion efficiency. Hysteresis is caused by complex current transient phenomena that become active during staircase voltammetry. Here, we address the root of this problem. We pinpoint the dynamic characteristics behind the slow transient responses to strategically predict a minimum time delay and thus estimate the power conversion efficiency under steady-state conditions. Circuit-element analysis and impedance spectroscopy confirm our predictions based on an advanced transient study. Our results fundamentally explore the possibility of reducing data time acquisition in a reliable performance assessment, providing disruptive solutions and perspectives toward systematic production of photovoltaic perovskites.

A Semi-Markov Stochastic Model for Operational Reliability Assessment of Hybrid AC and LCC-VSC-based DC System with Remote Wind Farms

A Semi-Markov Stochastic Model for Operational Reliability Assessment of Hybrid AC and LCC-VSC-based DC System with Remote Wind Farms

Effect of overlapping and space between stages of a three-bladed double-stage Savonius hydrokinetic turbine for low flow speed perennial river application

ABSTRACT Savonius-like hydrokinetic turbine (SAHT) in low-speed flow streams of perennial rivers is not much explored. In this paper, a triple-bladed double-stage SAHT is designed, and its performance is investigated for low flow speeds within 0.45–0.65 m/s under the effect of blade overlapping (within 10–20% in each stage), turbine diameter (within 208–234 mm), and without and with space between the stages (5.0 mm) at different operating speed ratios. Different combinations of design and operating input parameters are studied to obtain a reliable assessment of SAHT performance. After analyzing the results, the best conditions of input parameters and output performance parameters are reported in their nondimensional forms for future use of the obtained results. The torque produced by the turbine increased with the applied brake load, leading to its maximum value at the highest brake load 0.83 times the maximum load. The highest coefficient of performance is obtained as 0.086 at a space 0.25 times the maximum space (i.e. 20 mm), overlapping 15%, brake load 0.83 times the maximum load, tip speed ratio 0.276. And for this, the corresponding power 0.85 times the maximum power (i.e. 0.201 W) at free-stream flow speed of 0.85 times the maximum flow speed (i.e. 0.65 m/s) is obtained. The present turbine with space and overlapping exhibits better performance than some available SAHT designs of single/double stage, two-blade/multi-blade with/without overlap configurations. The novelty of the work is that an improved SAHT has been developed for low-flow speed perennial river applications in the band of low tip speed ratios by combining design conditions like double-stage, space between stages, and overlapping.

Reliability analysis for complex electromechanical multi-state systems utilizing universal generating function techniques

The large-scale adoption of complex electromechanical systems (CMESs) renders it challenging to perform reliability assessments for such systems. In terms of the reliability evaluation of CMESs, previous studies generally assume that only binary states exist within the systems. However, owing to the stochastic failures and various operating characteristics of the system components, not only the good/bad states but also the performance levels within the system should be taken into account when analyzing the reliability of CMESs. Hence, we abstract the CMES as a multi-layer and multi-state network in which minimum maintenance units (MMUs) and three types of connections constitute nodes and edges in different layers. Furthermore, multi-state models for different MMUs in the CMES are developed based on universal generating functions (UGFs). Aggregation operators are then utilized to aggregate these UGFs to obtain the multi-state models. Subsequently, a cascading failure model is introduced to incorporate MMUs into the system reliability assessment. Thus, the effects of the operating mechanism and topological attributes can be considered in the reliability evaluation of the CEMS. Finally, the CRHX high-speed train system is used as an example to demonstrate the applicability and effectiveness of the proposed method in the reliability assessment of the CMES.

Prediction of nonlinear seismic demand of inter-story isolated systems using improved multi-modal pushover analysis procedures

The engineering community requires a simple and engineering-friendly procedure to evaluate the seismic performance of inter-story isolated structures (IIS), considering the widespread application of IIS and the limitations of current static nonlinear analysis procedures. This paper proposes an improved multi-modal pushover analysis procedure (IMMPA) that is compatible with the code design spectrum and can account for the effects of higher mode responses and the strongly nonlinear nature of IIS. Nonlinear response history analysis (NRHA) is unnecessary for a reliable assessment of IIS seismic performance when IMMPA is utilized. The IMMPA involves converting the IIS into a linear non-proportional damping structure to satisfy performance requirements, decoupling it into modal oscillators with the equivalent of the nonlinear structure in the modal coordinates using complex-mode-superposition approaches, adjusting the input response spectrum excitation with its equivalent parameters, and converting the response spectrum excitation into the demand spectrum. The main modal oscillators are identified by the complex modal participation mass ratio; the respective acceleration is used as the weight factor of the lateral pushing force; and the capacity spectrum procedure is employed to determine the performance point. Combining the main modal responses using a complex-mode superposition complete square combination rule (CCQC) produces an effective nonlinear peak response. Finally, the IMMPA procedure is demonstrated using an IIS example and evaluated with NRHA data. The evaluation demonstrated that the IMMPA procedure can overcome conventional pushover limitations and efficiently evaluate IIS seismic requirements.

Patient-Reported Outcome-Based Performance Measures in Alternative Payment Models: Current Use, Implementation Barriers, and Principles to Succeed

ObjectivesPatient-reported outcome (PRO)-based performance measures (PRO-PMs) offer opportunities to aggregate survey data into a reliable and valid assessment of performance at the entity-level (eg, clinician, hospital, and accountable care organization). Our objective was to address the existing literature gap regarding the implementation barriers, current use, and principles for PRO-PMs to succeed. MethodsAs quality measurement experts, we first highlighted key principles of PRO-PMs and how alternative payment models (APMs) may be integral in promoting more widespread use. In May 2023, we reviewed the Centers for Medicare and Medicaid Services (CMS) Measures Inventory Tool for active PRO-PM usage within CMS programs. We finally present principles to prioritize as part PRO-PMs succeeding within APMs. ResultsWe identified 5 implementation barriers to PRO-PM use: original development of instrument, response rate sufficiency, provider burden, hesitancy regarding fairness, and attribution of desired outcomes. There existed 54 instances of active PRO-PM usage across CMS programs, including 46 unique PRO-PMs within 14 CMS programs. Five principles to prioritize as part of greater PRO-PM development and incorporation within APMs include the following: (1) clinical salience, (2) adequate sample size, (3) meaningful range of performance among measured entities and the ability to detect performance change in a reasonable time frame, (4) equity focus, and (5) appropriate risk adjustment. ConclusionsIdentified barriers and principles to prioritize should be considered during PRO-PM development and implementation phases to link available and novel measures to payment programs while ensuring provider and stakeholder engagement.

Non-destructive detection of high-strength wind turbine bolt looseness using digital image correlation

Looseness of high-strength wind turbine bolts is one of the main types of mechanical failure that threaten the quality and safety of wind turbines, and how to non-destructively detect bolt loosening is essential to accurate assessment of operational reliability of wind turbine structures. Therefore, to address the issue of looseness detection of high-strength wind turbine bolts, this paper proposes a non-destructive detection method based on digital image correlation (DIC). Firstly, the mathematical relationships between the inplane displacement component of the bolt’s nut surface, the bolt’s preload force loss and the bolt loosening angle are both deduced theoretically. Then, experimental measurements are respectively conducted with DIC with different small bolt loosening angles. The results show that the bolt loosening angle detection method based on DIC has a detection accuracy of over 95%, and the bolt’s preload force loss evaluated by the deduced relationship has a good agreement with the empirical value. Therefore, the proposed DIC-based bolt loosening angle detection method can meet the requirements of engineering inspection, and can achieve quantitative assessment of preload forces loss of wind turbine bolt.