Performance Assessment Techniques Research Articles

Wiener model based GMVC design considering sensor noise and delay

Even though there is a plethora of literature available for assessing linear control loop performance, they cannot be applied to the nonlinear control loops. In this paper, a nonlinear generalized minimum variance (NGMV) controller based on a single input–single output (SISO) Wiener model is proposed. The NGMV controller’s performance is used as a benchmark for a class of nonlinear control loops. The advantage of the proposed method is ability of online parameter estimation of the nonlinear model using common recursive least squares (RLS) method. In real-world applications, sensor and measurement tools force noises and extra delay to the control loop which poses limitations on achievable control performance. Hence, the classic control performance assessment techniques, is not attainable anymore. To handle the limitation caused by sensor delay, the k-step ahead prediction method is utilized. Further, the exponential digital filter is used in order to attenuate impact of the measurement noise on the controller. To show the effectiveness of the proposed method, a simulation test on a pH neutralization process is carried out.

Comparison of the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) to a Single-Item Performance Score

ABSTRACTConstruct: We compared a single-item performance score with the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) for their ability in assessing surgical competency. Background: Surgical programs are adopting competency-based frameworks. The adoption of these frameworks for assessment requires tools that produce accurate and valid assessments of knowledge and technical performance. An assessment tool that is quick to complete could improve feasibility, reduce delays, and result in a higher volume of assessments of learners. Previous work demonstrated that the 9-item O-SCORE can produce valid results; the goal of this study was to determine if a single-item performance rating (Is candidate competent to independently complete procedure: yes or no) completed at a separate viewing would correlate to the O-SCORE, thus increasing feasibility of procedural competence assessment. Approach: Nineteen residents and 2 staff orthopedic surgeons from the University of Ottawa volunteered for a 2-part OSCE-style station including a written questionnaire and videotaped simulated open reduction and internal fixation midshaft radius fracture. Each performance was rated independently by 3 orthopedic surgeons using a single-item performance score (Time 1). The performances were assessed again 6 weeks later by the 3 raters using the O-SCORE (Time 2). Correlation between the single-item performance score and the O-SCORE were evaluated. Results: Three orthopedic surgeons completed 21 ratings each resulting in 63 orthopedic ratings. There was a high level of correlation and agreement between the single-item performance score at Time 1 and Time 2 (κ correlation =0.72–1.00; p < .001; percentage agreement =90%–100%). The reliability of the O-SCORE at Time 2 with three raters was 0.83 and the internal consistency was 0.89. There was a tendency for each rater to assign more yes responses to the more senior trainees. Conclusions: A single-item performance score correlated highly with the O-SCORE in an orthopedic setting. A single-item score could be used to supplement a multi-item score with similar results in orthopedics. There is still benefit in completing multi-item scores such as the O-SCORE evaluations to guide specific areas of improvement and direct feedback.

Flexible budgeting influence on organizational inertia and flexibility

PurposeThe purpose of this paper is to investigate how the use of flexible budgets may influence different institutional logics (organizational inertia and flexibility).Design/methodology/approachA qualitative research based on a single case study in a multinational subsidiary company was carried out. The data were mainly collected using the dialog technique through open-ended and semi-structured interviews and complemented with direct observation in informal and formal meetings and the analysis of internal documents. Content analysis was used for the analysis of the findings.FindingsThe use of flexible budgets, which isolates the negative variations due to the decrease in sales volume, may contribute to organizational inertia. However, this can be counterbalanced if the managers try to minimize the decline in performance through initiatives that promote organizational flexibility. In this case study, it was found that the alignment between the production director and the controller, who frequently work under different institutional logics, was important to stimulate organizational flexibility particularly in continuous improvement projects.Research limitations/implicationsThe findings of this paper are based on only one in-depth case study. Hence, the results cannot be generalized, but a theoretical contribution can be made. Furthermore, the findings are constrained by the constructs used and the specific managerial and theoretical perspectives that have supported the analysis.Practical implicationsThese results can be useful particularly for companies that are dealing with the abrupt drop in the sales volume and use the flexible budget as a performance assessment technique. These firms must pay attention because this combination can stimulate organizational inertia. To counteract this problem, it is necessary that controllers and the managers work by understanding the initiatives that promote organizational flexibility, mainly by Kaizen projects, which can minimize performance decline.Social implicationsThe main contribution may be how to deal with the different managers’ behaviors, given the decrease in sales volume, and it can help an organization survives in times of economic recession and fierce competition environments.Originality/valueThis paper contributes to both practical and academic dimensions. Indeed, despite being widely used, flexible budgeting is not a widely researched topic.

Hierarchical Multiblock T-PLS Based Operating Performance Assessment for Plant-Wide Processes

With the pursuit of profit, the operating performance assessment of industrial processes has gradually attracted attention and concentration in modern industrial production. However, owing to the complexity of plant-wide processes, many existing operating performance assessment techniques are lacking in interpretation of the assessment result, and the cause identification is also difficult for the nonoptimal operating performance. In this study, a new operating performance assessment method is proposed for plant-wide processes based on hierarchical multiblock total projection to latent structures (T-PLS). The proposed approach divides the process variables into several conceptually meaningful blocks and incorporates block information into high level for operating performance assessment of plant-wide processes. With the new method, the assessment task can be greatly reduced and the interpretability of the assessment models are greatly improved. Once the process operating performance is nonoptimal, the resp...

The potential value of surrogate performance markers at caesarean section for the assessment of surgical competence

ObjectivesWe hypothsised that surrogate markers of competence (procedure time and estimated surgical blood loss) could be of value when assessing competence for trainees learning to undertake lower segment caesarean section (LSCS). The study aim was to investigate differences in surrogate markers of surgical competence at LSCS between different training grades of primary surgeon and for different grades of surgical assistant; to assess the potential value of using these surrogate markers in overall assessments of surgical competence. Study designOperating records for elective LSCS were reviewed from January 2007 to January 2015. Uncomplicated LSCS where a junior trainee (ST2/ST3) or a senior trainee (ST6+) was the primary surgeon were included. Inclusion criteria were LSCS with body mass index less than 35, singleton pregnancy, greater than 34 weeks’ gestation, no more than one previous caesarean section and no low-lying placenta. Procedure time, estimated blood loss and the effect of the grade of surgical assistant (consultant or trainee) on these were compared overall for all surgeons at ST2 level, ST3 level and senior level (ST6+) using Student’s t test and effect size calculation (Cohen test). ResultsDuring the study period 3099 uncomplicated LSCS were performed by 60 trainees: 1367 by ST2, 1085 by ST3 and 646 by senior trainees (ST6+). Mean procedure times and recorded estimated blood loss decreased significantly for each level of training (p < 0.05). The degree of experience of the surgical assistant was also associated with reduced procedure times and blood loss, this was significant for trainees in ST2 when assisted by a consultant (p < 0.05). ConclusionsTraining progression is associated with a reduction in procedure time and estimated blood loss, which are both surrogate markers of competence at LSCS. Current objective structured assessments of technical competence for trainees (OSATS) do not take these metrics into account when assessing performance. An enhanced assessment of trainees undertaking LSCS is suggested, reviewing both objective structured assessments of technical performance (OSATS) and surrogate markers of performance recorded in a surgical logbook to provide a more comprehensive assessment of overall competence.

Building Inspection System for Evaluating the Technical Performance of Existing Buildings

AbstractThe technical performance assessment of a building is essential for planning maintenance actions to improve its condition and functionality. The aim of this study is to present a building i...

Robotic Surgery Improves Technical Performance and Enhances Prefrontal Activation During High Temporal Demand

Robotic surgery may improve technical performance and reduce mental demands compared to laparoscopic surgery. However, no studies have directly compared the impact of robotic and laparoscopic techniques on surgeons’ brain function. This study aimed to assess the effect of the operative platform (robotic surgery or conventional laparoscopy) on prefrontal cortical activation during a suturing task performed under temporal demand. Eight surgeons (mean age ± SD = 34.5 ± 2.9 years, male:female ratio = 7:1) performed an intracorporeal suturing task in a self-paced manner and under a 2 min time restriction using conventional laparoscopic and robotic techniques. Prefrontal activation was assessed using near-infrared spectroscopy, subjective workload was captured using SURG-TLX questionnaires, and a continuous heart rate monitor measured systemic stress responses. Task progression scores (au), error scores (au), leak volumes (mL) and knot tensile strengths (N) provided objective assessment of technical performance. Under time pressure, robotic suturing led to improved technical performance (median task progression score: laparoscopic suturing = 4.5 vs. robotic suturing = 5.0; z = − 2.107, p = 0.035; median error score: laparoscopic suturing = 3.0 mm vs. robotic suturing = 2.1 mm; z = − 2.488, p = 0.013). Compared to laparoscopic suturing, greater prefrontal activation was identified in seven channels located primarily in lateral prefrontal regions. These results suggest that robotic surgery improves performance during high workload conditions and is associated with enhanced activation in regions of attention, concentration and task engagement.

Thermal Ablation of T1c Renal Cell Carcinoma: A Comparative Assessment of Technical Performance, Procedural Outcome, and Safety of Microwave Ablation, Radiofrequency Ablation, and Cryoablation

PurposeTo evaluate perioperative outcomes of thermal ablation with microwave (MW), radiofrequency (RF), and cryoablation for stage T1c renal cell carcinoma (RCC). Materials and MethodsA retrospective analysis of 384 patients (mean age, 71 y; range, 22–88 y) was performed between October 2006 and October 2016. Mean radius, exophytic/endophytic, nearness to collecting system or sinus, anterior/posterior, and location relative to polar lines; preoperative aspects and dimensions used for anatomic classification; and centrality index scores were 6.3, 7.9, and 2.7, respectively. Assessment of pre- and postablation serum blood urea nitrogen, creatinine, and estimated glomerular filtration rate was performed to assess functional outcomes. Linear regression analyses were performed to compare sedation medication dosages among the three treatment cohorts. Univariable and multivariable logistic regression analyses were performed to compare rates of residual disease and complications among treatment modalities. ResultsA total of 437 clinical stage T1N0M0 biopsy-proven RCCs measuring 1.2–6.9 cm were treated with computed tomography (CT)–guided MW ablation (n = 44; 10%), RF ablation (n = 347; 79%), or cryoablation (n = 46; 11%). There were no significant differences in patient demographic or tumor characteristics among cohorts. Complication rates and immediate renal function changes were similar among the three ablation modalities (P = .46 and P = .08, respectively). MW ablation was associated with significantly decreased ablation time (P < .05), procedural time (P < .05), and dosage of sedative medication (P < .05) compared with RF ablation and cryoablation. ConclusionsCT-guided percutaneous MW ablation is comparable to RF ablation or cryoablation for the treatment of stage T1N0M0 RCC with regard to treatment response and is associated with shorter treatment times and less sedation than RF ablation or cryoablation. In addition, the safety profile of CT-guided MW ablation is noninferior to those of RF ablation or cryoablation.

Consensus views on the optimum training curriculum for advanced minimally invasive surgery: A delphi study

IntroductionThere has been a wide uptake in the use of Minimal Invasive Surgery (MIS) globally across different surgical specialties. Whilst evidence exists for a structured training curriculum for basic laparoscopic surgery, there is little agreement on a complete framework for an advanced MIS training curriculum, defining the essential elements of the curriculum including the optimal assessment methods. The aim of this study is to obtain a consensus on the essential elements of a training curriculum for advanced MIS. Materials and MethodsA Delphi study was carried out involving 57 international experts in advanced MIS across different surgical specialties. A three round survey was conducted to reach consensus on the essential domains of a curriculum. This included defining the learners, trainers and training centres; curriculum content and competency based assessment. ResultsUnanimous agreement was reached for the completion of basic laparoscopic training before entry into advanced training. A trainer should have reached competency in advanced MIS and attended a ‘Train the trainer’ course. The curriculum should be delivered as modular training, including a multi-modal approach with a structured clinical proctorship programme. Formative assessment was considered as an integral part of learning and should be performed using objective work based assessment tools such as global assessment scale (GAS) forms. Accreditation in advanced MIS can be achieved by objective assessment of technical performance of unedited videos in addition to key clinical performance outcomes. ConclusionA consensus on the framework of an advanced MIS training curriculum has been achieved defining the essential elements of entry criteria, selection of trainers and training units and curriculum content. Multimodal learning, clinical proctorship programme and competency based assessment are integral parts of the curriculum.

Developing an objective assessment of surgical performance from operating room video and surgical imagery

ABSTRACTAn unbiased, repeatable process for assessing operating room performance is an important step toward quantifying the relationship between surgical training and performance. Hip fracture surgeries offer a promising first target in orthopedic trauma because they are common and they offer quantitative performance metrics that can be assessed from video recordings and intraoperative fluoroscopic images. Hip fracture repair surgeries were recorded using a head-mounted point-of-view camera. Intraoperative fluoroscopic images were also saved. The following performance metrics were analyzed: duration of wire navigation, number of fluoroscopic images collected, degree of intervention by the surgeon's supervisor, and the tip-apex distance (TAD). Two orthopedic traumatologists graded surgical performance in each video independently using an Skills (OSATS). Wire navigation duration correlated with weeks into residency and prior cases logged. TAD correlated with cases logged. There was no significant correlation between the OSATS total score and experience metrics. Total OSATS score correlated with duration and number of fluoroscopic images. Our results indicate that two metrics of hip fracture wire navigation performance—duration and TAD—significantly differentiate surgical experience. The methods presented have the potential to provide truly objective assessment of resident technical performance in the OR.

Development of Authentic Assessment instruments for Critical Thinking skills in Global Warming with a Scientific Approach

&lt;p class="Abstract"&gt;This study aims to develop an &lt;em&gt;authentic assessment&lt;/em&gt; instrument to measure critical thinking skills in global warming learning and to describe the suitability, easiness, and usefulness of the use instruments which are developed base on the teacher’s opinion. The development design is carried out by Borg &amp;amp; Gall (2003) development model, which is conducted with seven stages: information gathering stage, planning stage, product development stage, product test stage, product revision stage, field trial stage, and final product. The test subjects are students and teachers in SMA Lampung Tengah by using &lt;em&gt;purposive sampling&lt;/em&gt; technique. Global warming learning using &lt;em&gt;authentic assessment&lt;/em&gt; consists of a series of learning activities, including observing, discussing, exploring, associating and communicating. The results show the &lt;em&gt;authentic assessment&lt;/em&gt; techniques global warming to measure and cultivate critical thinking skills consisting of written tests, performance, portfolios, projects, and attitudes. The developed assessment model meets content and constructs validity, and effectively improves students' critical thinking skills and has a high level of suitability, easiness, and usefulness well-being. The assessment techniques are used in global warming learning are performance assessment techniques, portfolios, projects, products, and attitude that together contribute to the improvement of critical thinking skills on 97,4% of global warming learning.&lt;/p&gt;

Quantitative evaluation of thin uranium foils using X-ray radiography

The quality of research reactor fuel plates fabricated for irradiation testing purposes is dependent on the distribution of uranium bearing fuel phase within the fuel meat. This distribution is routinely monitored in plate type fuel to avoid excessive variability in local power that can lead to ‘hot spots’ that increase the probability for fuel failure. X-ray imaging has been routinely used to successfully eliminate this failure mode. However, characterization of this variability at a much higher resolution is also critical to support fuel performance assessment and traditional analysis techniques are not sufficient to meet research needs. An accurate and reproducible method for using X-ray imaging to non-destructively locate the distributed fuel phase and to evaluate the local uranium density in a fuel plate has been developed and is discussed in this paper. Four mini research reactor fuel plats were radiographed along with a well characterized U-8Mo step wedge using identical radiography exposure conditions. The intensity distribution inside the foil were then correlated with the intensity values of the U-8Mo step wedge. Considering the attenuation and density, the spatial distribution of effective fuel thickness was estimated and eventually the distribution of average fuel loading for each fuel plate was estimated. Results demonstrate that the X-ray imaging can be utilized to estimate fuel loading variation in the fuel plate samples and can serve as a representative method for the assessment of nuclear fuels plates for research purposes.

Fatores associados à eficácia no desempenho das ações técnico-táticas do voleibol

DOI: http://dx.doi.org/10.5007/1980-0037.2017v19n1p74 The aim of the study was to investigate the factors associated to performance efficacy of technical-tactical actions in volleyball. Based on the protocol of the Instrument for Technical and Tactical Performance Assessment in Volleyball (IAD-VB), 44.025 match actions performed by male and female athletes from u-15, u-16 and u-17 finalist teams of the 2010 Santa Catarina Volleyball Championship in Brazil were observed. Data were analyzed on SPPS 21 software using the logistic regression technique, adopting 5% significance level for the interpretation of results. Adjusted model explained 51.40% efficacy variance, and the factors associated with this variable were decision making, adjustment, efficiency, and competition level.

Data mining and control loop performance assessment: The multivariate case

Control loop performance assessment (CLPA) techniques assume that the data being analyzed is generated during steady state operation with fixed plant dynamics and controller parameters. However, in industrial settings one often encounters environmental and feedstock variations which can induce significant changes in the plant dynamics. Availability of data sets corresponding to fixed configurations is therefore questionable in industrial scenarios, in which case it becomes imperative to extract the same from routine plant operating data. This article proposes a technique for segmenting multivariate control loop data into portions corresponding to fixed steady state operation of the system. The proposed technique exploits the fact that changes in the operating region of the system lead to changes in variance‐covariance matrix of multivariate control loop data. The univariate interval halving technique is fused with Mahalanobis distance to develop a multivariate tool that accounts for interactions between variables. The resulting data segments can be used for reliable CLPA and/or for user defined benchmarking of control loops. A multivariate control loop performance index is also proposed that requires significantly less data as compared to one of the previously proposed techniques. The proposed technique requires only routine operating data from the plant, and is tested on benchmark systems in the literature with simulations. Experimental validation on a model predictive control system aimed at maintaining the temperature profile of a metal plate demonstrates applicability of the technique to industrial systems. The proposed technique acts as a tool for preprocessing data relevant to CLPA and can be applied to large scale interacting multivariate systems. © 2017 American Institute of Chemical Engineers AIChE J, 63: 3311–3328, 2017

Technical Performance and Economic Feasibility Assessment of A Mechanical Yam Harvester

Technical Performance and Economic Feasibility Assessment of A Mechanical Yam Harvester

Smoothness of surgical tool tip motion correlates to skill in endovascular tasks

Current performance assessment techniques in endovascular surgery are subjective or limited to grading scales based solely on an expert's observation of a novice's task execution. Since most endovascular procedures involve performing fine motor control tasks that require complex dexterous movements, this paper evaluates objective and quantitative metrics of performance that capture movement quality through the computation of tool tip movement smoothness. An experiment was designed that involved recording the catheter tip movement from 20 subjects performing four fundamental endovascular tasks in each of three sessions using manual catheterization on a physical model and in a simulation environment. Several motion-based performance measures that have been shown to reliably assess skill in other domains were computed and tested for correlation with subjective data that were simultaneously obtained from the global rating scale assessment tool. Metrics that captured movement smoothness produced statistically significant correlations with the observation-based assessment metrics and were able to differentiate skill among participants. In particular, submovement analysis led to metrics that captured statistically significant differences across ability group, session, experimental platform, and task. Objective and quantitative metrics that capture movement smoothness could be incorporated into future training protocols to provide detailed feedback on trainee performance.

Monitoring Methodology of Interventions for Riverbanks Stabilization: Assessment of Technical Solutions Performance

The success of the river rehabilitation process depends not only on the intervention project but also on the attention given to the monitoring and evaluation of the implemented measures. In spite of the recent efforts on the quantification and sharing of interventions’ results, there are still very few written records documenting that information. The present work proposes a monitoring methodology for riverbanks’ stabilization interventions by comparing the results obtained by monitoring the implemented technical solutions with the corresponding theoretical time-expected performance. For this purpose, two case studies located in the south of Portugal were analysed. For each of those case studies, the dominant factors that influenced the temporal evolution of the observed outcomes were assessed. It was possible to verify that, depending on the implemented technical solution type and on the specific characteristics of the intervention site, monitoring actions can be fundamental to achieve the expected technical and ecological efficiency, namely when bioengineering technical solutions are involved. This work led to the development of a database structure that will gather information on the application of the proposed methodology over time and that shall be improved with the analysis of the behaviour of other riverbanks’ stabilization interventions. The development of this database will surely contribute to enable choosing and applying the most adequate riverbank protection solutions.

Application of objective clinical human reliability analysis (OCHRA) in assessment of technical performance in laparoscopic rectal cancer surgery.

Laparoscopic rectal resection is technically challenging, with outcomes dependent upon technical performance. No robust objective assessment tool exists for laparoscopic rectal resection surgery. This study aimed to investigate the application of the objective clinical human reliability analysis (OCHRA) technique for assessing technical performance of laparoscopic rectal surgery and explore the validity and reliability of this technique. Laparoscopic rectal cancer resection operations were described in the format of a hierarchical task analysis. Potential technical errors were defined. The OCHRA technique was used to identify technical errors enacted in videos of twenty consecutive laparoscopic rectal cancer resection operations from a single site. The procedural task, spatial location, and circumstances of all identified errors were logged. Clinical validity was assessed through correlation with clinical outcomes; reliability was assessed by test-retest. A total of 335 execution errors identified, with a median 15 per operation. More errors were observed during pelvic tasks compared with abdominal tasks (p<0.001). Within the pelvis, more errors were observed during dissection on the right side than the left (p=0.03). Test-retest confirmed reliability (r=0.97, p<0.001). A significant correlation was observed between error frequency and mesorectal specimen quality (r s=0.52, p=0.02) and with blood loss (r s=0.609, p=0.004). OCHRA offers a valid and reliable method for evaluating technical performance of laparoscopic rectal surgery.

A decade of RAPID—Reflections on the development of an open source geoscience code

AbstractEarth science increasingly relies on computer‐based methods and many government agencies now require further sharing of the digital products they helped fund. Earth scientists, while often supportive of more transparency in the methods they develop, are concerned by this recent requirement and puzzled by its multiple implications. This paper therefore presents a reflection on the numerous aspects of sharing code and data in the general field of computer modeling of dynamic Earth processes. Our reflection is based on 10 years of development of an open source model called the Routing Application for Parallel Computation of Discharge (RAPID) that simulates the propagation of water flow waves in river networks. Three consecutive but distinct phases of the sharing process are highlighted here: opening, exposing, and consolidating. Each one of these phases is presented as an independent and tractable increment aligned with the various stages of code development and justified based on the size of the users community. Several aspects of digital scholarship are presented here including licenses, documentation, websites, citable code and data repositories, and testing. While the many existing services facilitate the sharing of digital research products, digital scholarship also raises community challenges related to technical training, self‐perceived inadequacy, community contribution, acknowledgment and performance assessment, and sustainable sharing.

Northwest ductless heat pump Pilot Project—a holistic approach to market transformation and evaluation

There are approximately one million existing electrically heated homes in the Northwest. The Northwest region of the USA as defined for this project includes the states of Washington, Oregon, Idaho, and western Montana. Within this area, there are a range of climate zones. The region also includes a wide range of electric markets, from rural to large urban areas. Ductless heat pumps (DHPs) have the potential to save several hundred average megawatts in this market and reduce the carbon intensity of target homes by more than 30 %. To leverage this large potential resource for the region, the Northwest Energy Efficiency Alliance (NEEA) used a holistic approach to design, implement, and evaluate the Northwest DHP Pilot Project. NEEA worked with contractors, manufacturers, distributors, and utilities to generate demand for DHPs, build a thriving and sustainable network to meet that demand, and ultimately transform the DHP market. NEEA designed the DHP Pilot Project evaluation to conduct an integrated assessment of DHP technical performance and market acceptance. The evaluation included five “tiers” of DHP research: (1) market acceptance, including interviews with manufacturers, distributors, utilities, program implementation staff, and nearly 300 participants; (2) lab testing of two DHPs; (3) field monitoring of 95 participating homes; (4) billing analysis of the pilot population (3899 sites); and (5) cost-effectiveness analysis of the DHP measure. By taking a “360 degree” perspective, the evaluators were able to make granular distinctions between performance-based and behavior-based determinants of energy savings. This paper discusses the tools used to influence DHP market perception and to overcome skeptics of this technology. The paper also assesses the evaluation approach, including the pros and cons of assessing a technology and program design from multiple angles in order to identify reliable savings and the potential for future programs. The paper summarizes extensive primary research and case studies. As such, the paper focuses solely on the direct experience and findings from the research.