Objective Assessment Of Performance Research Articles

Developing a coaching mechanism for practicing surgeons

While performance feedback and assessment are hallmarks of surgical training, they abruptly cease after training is completed. In their absence, performance may stagnate and poor habits persist. Our aim was to develop a coaching mechanism for practicing surgeons with feedback provision based on objective performance assessment. Technical and nontechnical intraoperative video recordings from laparoscopic or robotic cholecystectomies, colectomies, and hysterectomies were assessed by a blinded surgeon and a human factors expert, respectively. Aspects of performance in need of improvement were noted, and a coaching session was developed for feedback provision to participating surgeons. This 4-hour coaching session consisted of a didactic lecture with video review and hands-on practice using procedural and mannequin-based simulation. Thirty-two practicing surgeons (18 general; 14 gynecologists) from 6 different hospitals were assessed, and 9 of them participated in coaching. Technical aspects identified for performance improvement included suboptimal trocar placement, inadequate critical view achievement during laparoscopic cholecystectomies, poor visualization of the operating field, bimanual dexterity, and dissection techniques, while nontechnical aspects included inappropriate handling of distractions and interruptions, poor ergonomic positioning and situational awareness, and inadequate mitigation of delays. Most surgeons appropriately accomplished some of the objectives of the distraction scenario, but none was able to achieve expert levels on Fundamentals of Laparoscopy tasks. Participants perceived the coaching sessions as highly valuable. Our study identified several technical and nontechnical skill sets of practicing surgeons in need of improvement and provided support for the implementation of coaching programs for surgeons on an ongoing basis.

Assessing Wire Navigation Performance in the Operating Room

There are no widely accepted, objective, and reliable tools for measuring surgical skill in the operating room (OR). Ubiquitous video and imaging technology provide opportunities to develop metrics that meet this need. Hip fracture surgery is a promising area in which to develop these measures because hip fractures are common, the surgery is used as a milestone for residents, and it demands technical skill. The study objective is to develop meaningful, objective measures of wire navigation performance in the OR. Resident surgeons wore a head-mounted video camera while performing surgical open reduction and internal fixation using a dynamic hip screw. Data collected from video included: duration of wire navigation, number of fluoroscopic images, and the degree of intervention by the surgeon׳s supervisor. To determine reliability of these measurements, 4 independent raters performed them for 2 cases. Raters independently measured the tip-apex distance (TAD), which reflects the accuracy of the surgical placement of the wire, on all the 7 cases. University of Iowa Hospitals and Clinics in Iowa City, IA-a public tertiary academic center. In total 7 surgeries were performed by 7 different orthopedic residents. All 10 raters were biomedical engineering graduate students. The standard deviations for anteroposterior, lateral, and combined TAD measurements of the 10 raters were 2.7, 1.9, and 3.7mm, respectively, and interrater reliability produced a Cronbach α of 0.97. The interrater reliability analysis for all 9 video-based measures produced a Cronbach α of 0.99. Several video-based metrics were consistent across the 4 video reviewers and are likely to be useful for performance assessment. The TAD measurement was less reliable than previous reports have suggested, but remains a valuable metric of performance. Nonexperts can reliably measure these values and they offer an objective assessment of OR performance.

Catheter manipulation analysis for objective performance and technical skills assessment in transcatheter aortic valve implantation.

PurposeTranscatheter aortic valve implantation (TAVI) demands precise and efficient handling of surgical instruments within the confines of the aortic anatomy. Operational performance and dexterous skills are critical for patient safety, and objective methods are assessed with a number of manipulation features, derived from the kinematic analysis of the catheter/guidewire in fluoroscopy video sequences.MethodsA silicon phantom model of a type I aortic arch was used for this study. Twelve endovascular surgeons, divided into two experience groups, experts (n=6) and novices (n=6), performed cannulation of the aorta, representative of valve placement in TAVI. Each participant completed two TAVI experiments, one with conventional catheters and one with the Magellan robotic platform. Video sequences of the fluoroscopic monitor were recorded for procedural processing. A semi-automated tracking software provided the 2D coordinates of the catheter/guidewire tip. In addition, the aorta phantom was segmented in the videos and the shape of the entire catheter was manually annotated in a subset of the available video frames using crowdsourcing. The TAVI procedure was divided into two stages, and various metrics, representative of the catheter’s overall navigation as well as its relative movement to the vessel wall, were developed.ResultsExperts consistently exhibited lower values of procedure time and dimensionless jerk, and higher average speed and acceleration than novices. Robotic navigation resulted in increased average distance to the vessel wall in both groups, a surrogate measure of safety and reduced risk of embolisation. Discrimination of experience level and types of equipment was achieved with the generated motion features and established clustering algorithms.ConclusionsEvaluation of surgical skills is possible through the analysis of the catheter/guidewire motion pattern. The use of robotic endovascular platforms seems to enable more precise and controlled catheter navigation.

Contrast enhancement via texture region based histogram equalization

This paper presents two novel contrast enhancement approaches using texture regions-based histogram equalization (HE). In HE-based contrast enhancement methods, the enhanced image often contains undesirable artefacts because an excessive number of pixels in the non-textured areas heavily bias the histogram. The novel idea presented in this paper is to suppress the impact of pixels in non-textured areas and to exploit texture features for the computation of histogram in the process of HE. The first algorithm named as Dominant Orientation-based Texture Histogram Equalization (DOTHE), constructs the histogram of the image using only those image patches having dominant orientation. DOTHE categories image patches into smooth, dominant or non-dominant orientation patches by using the image variance and singular value decomposition algorithm and utilizes only dominant orientation patches in the process of HE. The second method termed as Edge-based Texture Histogram Equalization, calculates significant edges in the image and constructs the histogram using the grey levels present in the neighbourhood of edges. The cumulative density function of the histogram formed from texture features is mapped on the entire dynamic range of the input image to produce the contrast-enhanced image. Subjective as well as objective performance assessment of proposed methods is conducted and compared with other existing HE methods. The performance assessment in terms of visual quality, contrast improvement index, entropy and measure of enhancement reveals that the proposed methods outperform the existing HE methods.

Objective Assessment of Students Performance before and After Joining Medical College

Objective Assessment of Students Performance before and After Joining Medical College

Performance Assessment of TMOs

High dynamic range (HDR) imaging has gained momentum with the growth in digital photography technology. In this paper, performance assessment of various tone mapping techniques has been presented. HDR images were used and Tone Mapping Operators (TMOs) such as Gamma Correction, Reinhard, Drago, Ashkmin were applied to generate Low Dynamic Range (LDR) images. After that, LDR images are rated using objective assessment parameters such as luminance, mean, MSE and mPSNR. A new modified Ashikmin TMO is reported in this work. The results have shown that modified Ashikhmin has generated better LDR image than other TMOs. In addition, the profiles of objective performance assessment were also included to compare the usability of TMOs for generating better LDR images.

자료 포락 분석 방법을 이용한 객관적 건물 에너지 성능 평가

Objective energy performance assessment of buildings is crucial for building stakeholders' rational decision making. One of the most popular building energy performance measures is Energy Use Intensity (EUI, kwh/m2.yr). This has been widely used since it is straightforward, simple and easy to understand. However, it has a severe drawback that it only shows the number of consumed energy per unit floor area and can't represent objective energy performance of a building. In other words, EUI does not deliver how well a building serves occupants and provides satisfactory services (e.g. thermal comfort). It is often misinterpreted in a way that the less EUI, the better building energy performance is. In this paper, a Data Envelopment Analysis (DEA) was applied to assess objective building energy performance. The DEA quantifies performance of a given system when multiple inputs and outputs are given. The DEA is a data-oriented and non-parametric method. Thus, it does not require any energy model and can consider multivariate inputs/outputs simultaneously. For the study, a number of virtual buildings were generated out of Monte-Carlo sampling and then simulated using EnergyPlus to derive a data set. Energy consumption was used as an input and building service levels (e.g. occupancy density [people/m2], operation time [hrs/yr], thermal comfort [PPD]) were used as outputs. It is shown that the DEA is a more objective and rational performance assessment method than the EUI, and can be a good alternative for building energy performance evaluation.

Influence of school community and fitness on prevalence of overweight in Australian school children

The study objectives were (1) to determine the variation in prevalence of overweight between school communities, (2) to evaluate the relationship between cardiorespiratory fitness and the probability of being overweight among different school communities, and (3) to test whether this relationship varies between school communities. Using a repeated cross-sectional design, data from 31,424 (15,298 girls, 16,126 boys) Australian school children who had objective assessments of body composition and physical performance were used. Ninety-one schools located across 5 states and territories were included. Independent samples were taken across 12 school years (2000–2011). Analysis used generalised linear mixed models in R with a two-level hierarchical structure—children, nested within school communities. Predictor variables considered were: level 1—gender, age, cardiorespiratory fitness and year of measurement; level 2—school community. A total of 24.6% of the children were overweight and 69% were of low fitness. Variation in the prevalence of overweight between school communities was significant, ranging from 19% to 34%. The probability of being overweight was negatively associated with increasing cardiorespiratory fitness. The relationship was steepest at low fitness and varied markedly between school communities. Children of low fitness had probabilities of being overweight ranging between 26% and 75% depending on school community, whereas those of high fitness had probabilities of <2%. Our findings suggest that most might be gained from a public health perspective by focusing intervention on the least fit children in the worst-performing communities.

Cross-site strain comparison of pharmacological deficits in the touchscreen visual discrimination test.

The low rate of success for identifying effective treatments for cognitive dysfunction has prompted recent efforts to improve pharmaceutical discovery and development. In particular, investigators have emphasized improving translation from pre-clinical to clinical research. A specific area of focus has been touchscreen technology; this computer-automated behavioral testing method provides an objective assessment of performance that can be used across species. As part of a larger multi-site study with partners from the Innovative Medicines Initiative (IMI), two US sites, AbbVie and Pfizer, conducted a cross-site experiment with a common protocol for the visual discrimination (VD) task using identical testing equipment, stimuli, and rats of the same strains, sex, and age from the same supplier. As most touchscreen-based rodent experiments have used Lister-Hooded rats that are not readily available outside of Europe, a strain comparison with male Long-Evans rats was conducted as part of the study. Rats were trained for asymptotic performance, and test sessions were performed once per week in a full crossover design with cognition-impairing drugs. Drugs tested were phencyclidine and S-ketamine (N-methyl-D-aspartate (NMDA) antagonists), D-amphetamine (indirect dopamine agonist), and scopolamine (muscarinic antagonist). Satellite brain and plasma samples were taken to confirm appropriate exposures. Results indicate that both rat strains show similar patterns of impairment, although Lister-Hooded rats were more sensitive than Long-Evans rats to three out of four drugs tested. This suggests that researchers should fully explore dose-response relationships in their strain of choice and use care in the interpretation of reversal of cognitive impairment.

Robotic and clinical evaluation of upper limb motor performance in patients with Friedreich's Ataxia: an observational study.

BackgroundFriedreich’s ataxia (FRDA) is the most common hereditary autosomal recessive form of ataxia. In this disease there is early manifestation of gait ataxia, and dysmetria of the arms and legs which causes impairment in daily activities that require fine manual dexterity. To date there is no cure for this disease. Some novel therapeutic approaches are ongoing in different steps of clinical trial. Development of sensitive outcome measures is crucial to prove therapeutic effectiveness. The aim of the study was to assess the reliability and sensitivity of quantitative and objective assessment of upper limb performance computed by means of the robotic device and to evaluate the correlation with clinical and functional markers of the disease severity.MethodsHere we assess upper limb performances by means of the InMotion Arm Robot, a robot designed for clinical neurological applications, in a cohort of 14 children and young adults affected by FRDA, matched for age and gender with 18 healthy subjects. We focused on the analysis of kinematics, accuracy, smoothness, and submovements of the upper limb while reaching movements were performed. The robotic evaluation of upper limb performance consisted of planar reaching movements performed with the robotic system. The motors of the robot were turned off, so that the device worked as a measurement tool. The status of the disease was scored using the Scale for the Assessment and Rating of Ataxia (SARA). Relationships between robotic indices and a range of clinical and disease characteristics were examined.ResultsAll our robotic indices were significantly different between the two cohorts except for two, and were highly and reliably discriminative between healthy and subjects with FRDA. In particular, subjects with FRDA exhibited slower movements as well as loss of accuracy and smoothness, which are typical of the disease. Duration of Movement, Normalized Jerk, and Number of Submovements were the best discriminative indices, as they were directly and easily measurable and correlated with the status of the disease, as measured by SARA.ConclusionsOur results suggest that outcome measures obtained by means of robotic devices can improve the sensitivity of clinical evaluations of patients’ dexterity and can accurately and efficiently quantify changes over time in clinical trials, particularly when functional scales appear to be no longer sensitive.

Preditores de Desempenho Escolar no 5º Ano do Ensino Fundamental

Student, family and school factors account for individual differences in school performance through elementary school (EF). The aim of this prospective study was to test a model to predict academic performance in the 5th year of EF, adopting academic skills, social skills, behavior problems and perceived school hassles in the third year as predictors, as well as the length of exposure to early childhood education – ECE and the socioeconomic background of the student population in each EF school. Participants were 248 students attending four public schools. The instruments used were: Brazil Economic Classification Criteria, Academic Performance Test, teacher Social Skills Rating System – SSRS, Inventory of School Hassles and a collective assessment of school performance. Academic skills, social skills, and perceived stress in the third year predicted further academic performance. EF school effects on objective performance assessment were different from their effects on the teacher’s judgment of academic competence.

If We Do Our Job Correctly, Nobody Gets Hurt by Nepotism

The term nepotism (and even worse, cronyism) carries a negative connotation: favoring a relative (or friend) in an employment situation without considering the individual's suitability for the job. Although there can be obvious benefits associated with hiring kin (e.g., a sense of trust, swift learning of job-relevant content, loyalty, etc.; Bellow, 2003; Jones &amp; Stout, 2015), the term itself implies that nepotism is a bad thing, and organizations often take steps to keep bad things from happening. Jones and Stout (2015) have argued that sweeping antinepotism policies in organizations eliminate the positives associated with hiring via a social connection preference, and such policies can lead to unfair discrimination. As industrial and organizational (I-O) psychologists, however, if we do our job—and by that I mean exemplary and objective screening, hiring, and performance assessment—and if we adequately manage the negative impressions that may reside in the minds of employees regarding nepotism, nobody gets hurt.

Objectivity and subjectivity in performance evaluation and autonomous motivation: An exploratory study

A considerable body of accounting research has analysed the effectiveness of subjective performance assessment to compensate for the shortcomings of objective performance measures. The present paper adds a further perspective to this research by analysing the interaction between autonomous motivation and performance evaluation systems that either contain only objective measures or also apply subjective assessment. Findings of one experiment indicate that evaluation systems that contain either only objective or both objective and subjective components do not exhibit a significant interaction with autonomous motivation and that mixed evaluation is superior independent of the level of autonomous motivation. However, the findings of a further experiment, investigating the impact of imprecision that can accompany subjective performance evaluation, indicate that, for people low in autonomous motivation, objective and precise performance assessment is superior to performance evaluation that also contains imprecise subjective components. In the case of people high in autonomous motivation, the opposite holds true.

Assessing growth performance and deficiency of climber species on tropical greenwalls

Vertical greening can contribute to urban green infrastructure, urban heat-island amelioration and climate adaptation. Tropical greenwall practice, hindered by inadequate scientific knowledge and experience, can benefit from objective assessment of species performance and growth deficiency. Comprehensive survey of critical climber-plant traits shortlisted 20 potentially suitable species. A climber-selection matrix was established to facilitate species selection and greenwall design. Perennial woody climbers with ornamental flower or foliage and biological potential to reach 13-m height were preferred. They represented two attachment modes, namely mesh-climber and concrete-(bare-wall)climber groups. A field study was designed to monitor their growth from seedlings in experimental plots. Growth conditions were optimized by providing high-quality soil mix and irrigation. The 32-month study period included three active-growing seasons with two interspersed slow-growing seasons. Field assessment methods were developed to acquire systematic data on plant performance indicators and deficiency symptoms. Key attributes were selected to compute climber performance index (CPI) and climber deficiency index (CDI) as synoptic representation of greenwall-application suitability. Mesh-climbers performed much better than concrete-climbers across all key performance indicators, and with notably better CPI. They demonstrated faster establishment and growth rates and more ornamental flowers. The two groups had similar CDI mainly due to insufficient foliage density and foliage loss. For mesh-climbers, Quisqualis indica, Wisteria sinensis and Lonicera japonica had excellent performance, and the remaining species were good to fair. For concrete-climbers, only Parthenocissus dalzielii was rated good and Campsis grandiflora fair, with the rest poor to very poor.

Is objective and accurate cognitive assessment across the menstrual cycle possible? A feasibility study.

Objectives:Variation in plasma hormone levels influences the neurobiology of brain regions involved in cognition and emotion processing. Fluctuations in hormone levels across the menstrual cycle could therefore alter cognitive performance and wellbeing; reports have provided conflicting results, however. The aim of this study was to assess whether objective assessment of cognitive performance and self-reported wellbeing during the follicular and luteal phases of the menstrual cycle is feasible and investigate the possible reasons for variation in effects previously reported.Methods:The Cambridge Neuropsychological Test Automated Battery and Edinburgh Postnatal Depression Scale were used to assess the cognitive performance and wellbeing of 12 women. Data were analysed by self-reported and hormone-estimated phases of the menstrual cycle.Results:Recruitment to the study and assessment of cognition and wellbeing was without issue. Plasma hormone and peptide estimation showed substantial individual variation and suggests inaccuracy in self-reported menstrual phase estimation.Conclusion:Objective assessment of cognitive performance and self-assessed wellbeing across the menstrual cycle is feasible. Grouping data by hormonal profile rather by self-reported phase estimation may influence phase-mediated results. Future studies should use plasma hormone and peptide profiles to estimate cycle phase and group data for analyses.

Hybrid data acquisition and processing strategies with increased throughput and selectivity: pSMART analysis for global qualitative and quantitative analysis.

Data-dependent acquisition (DDA) and data-independent acquisition strategies (DIA) have both resulted in improved understanding of proteomics samples. Both strategies have advantages and disadvantages that are well-published, where DDA is typically applied for deep discovery and DIA may be used to create sample records. In this paper, we present a hybrid data acquisition and processing strategy (pSMART) that combines the strengths of both techniques and provides significant benefits for qualitative and quantitative peptide analysis. The performance of pSMART is compared to published DIA strategies in an experiment that allows the objective assessment of DIA performance with respect to interrogation of previously acquired MS data. The results of this experiment demonstrate that pSMART creates fewer decoy hits than a standard DIA strategy. Moreover, we show that pSMART is more selective, sensitive, and reproducible than either standard DIA or DDA strategies alone.

Performance evaluation of DNA copy number segmentation methods.

A number of bioinformatic or biostatistical methods are available for analyzing DNA copy number profiles measured from microarray or sequencing technologies. In the absence of rich enough gold standard data sets, the performance of these methods is generally assessed using unrealistic simulation studies, or based on small real data analyses. To make an objective and reproducible performance assessment, we have designed and implemented a framework to generate realistic DNA copy number profiles of cancer samples with known truth. These profiles are generated by resampling publicly available SNP microarray data from genomic regions with known copy-number state. The original data have been extracted from dilutions series of tumor cell lines with matched blood samples at several concentrations. Therefore, the signal-to-noise ratio of the generated profiles can be controlled through the (known) percentage of tumor cells in the sample. This article describes this framework and its application to a comparison study between methods for segmenting DNA copy number profiles from SNP microarrays. This study indicates that no single method is uniformly better than all others. It also helps identifying pros and cons of the compared methods as a function of biologically informative parameters, such as the fraction of tumor cells in the sample and the proportion of heterozygous markers. This comparison study may be reproduced using the open source and cross-platform R package jointseg, which implements the proposed data generation and evaluation framework: http://r-forge.r-project.org/R/?group_id=1562.

Transforming nursing education: Development and evaluation of interprofessional clinical skills training for students on clinical placement

Interprofessional education is viewed as necessary for students in disciplines such as medicine, nursing and allied health in preparation for the real world of collaborative practice. We describe the development, implementation and evaluation of a suite of interprofessional education sessions that were delivered to students in six pre-registration healthcare courses. While on clinical placement, students attended face to face interprofessional clinical education sessions. Interprofessional workshops, seminars, simulation education sessions and tutorials were conducted by clinical educators in addition to other on-site experiential learning. Student surveys (n = 746) and several interviews provided feedback. Students were enthusiastic about learning with other disciplines and gave positive feedback about interactive learning (98% were satisfied). They described knowledge improvement, better understanding of the roles and practices of others, as well as effective teaching strategies. Difficulties occurred in pairing students from more than one discipline to situate them together, because of unequal numbers and scheduling of placement dates. The current model, however, where IPL is delivered to students of multiple disciplines on the clinical placement sites is feasible, practical and desirable. Further work is needed to embed this teaching and learning model in curricula and to develop objective performance assessment.

Automatic Gear Shifting in Sport Motorcycles

Performing safe and effective gear shifts on two-wheeled vehicles in an automatic way is a challenging task, mainly due to the complexity of the system dynamics and the safety-critical aspects of the application. This work offers one of the first contributions in this area, analyzing all the aspects related to this control problem. Specifically, this work provides two main results: an objective quality assessment of the gear-shifting performance and a new gear shift control strategy that optimizes the tradeoff between duration and comfort. The results obtained on an instrumented vehicle confirm the suitability of the proposed approach.

Performance Variables and Professional Experience in Simulated Laparoscopy: A Two-Group Learning Curve Study

Virtual reality simulators are increasingly used in laparoscopy training. Such simulators allow objective assessment of performance. However, both low-level variables and overall scores generated by the simulator can be hard to interpret. We present a method to generate intermediate performance variables and show how the resulting variables can be used to investigate the development of laparoscopic skills. A beginner group (n = 16) and a group with intermediate laparoscopic experience (n = 9) participated in a 5-session, basic skills training course hosted by the Department of Technical Medicine at the University of Twente. Multiple simulator-generated variables were aggregated into 4 performance variables: duration, left-hand motion, right-hand motion, and damage. Differences in performance were analyzed in relation to proficiency values. Damage performance differentiated the most between groups and proficiency values; motion performance variables differentiated the least. The more experienced group outperformed the beginner group at damage by the end of the course. Differentiating between duration, left-hand motion, right-hand motion, and damage is a useful way to investigate laparoscopic performance development. Different performance variables follow different trajectories toward expertise. Valid and reliable clinical damage parameters are needed to investigate the relation of real-world damage to simulator damage.