Performance Evaluation Instrument Research Articles

Capturing bank employees’ subjectivity as a management technology: study of an performance evaluation instrument

Resumo Introdução: a dominação sobre os trabalhadores na fase neoliberal do capitalismo se desloca da sujeição dos corpos para o controle da subjetividade. Entre as tecnologias gerenciais que auxiliam esse processo, estão os instrumentos de avaliação de desempenho. Pela centralidade do sistema financeiro na sociedade capitalista, é em empresas desse ramo que costumam ocorrer inovações tecnológicas com maior potencial para se disseminar por toda a sociedade. Objetivo: para identificar a racionalidade subjacente a essas tecnologias, neste artigo estudamos o instrumento de avaliação de desempenho utilizado por um banco privado brasileiro. Métodos: foi realizada uma análise documental do manual explicativo do instrumento. Resultados: o instrumento, por meio de um algoritmo matemático e do uso de avaliação 360 graus, por um lado, determina um percentual mínimo de trabalhadores com desempenho insatisfatório, e, por outro, limita a porcentagem máxima de trabalhadores que poderão atingir o mais alto desempenho. Desse modo, os trabalhadores são estimulados permanentemente a concorrerem entre si para ocuparem posições predeterminadas. Conclusão: identificamos no instrumento um paradigma ilustrativo da racionalidade gerencial subjacente às mais diversas tecnologias gerenciais que, por meio do fomento à concorrência generalizada entre os trabalhadores, contribuem para um ambiente de trabalho adoecedor.

Is Robotic Console Time a Surrogate for Resident Operative Autonomy?

Robotic-assisted surgery is an increasing part of general surgery training, but resident autonomy on the robotic platform can be hard to quantify. Robotic console time (RCT), the percentage of time the resident controls the console, may be an appropriate measure of resident operative autonomy. This study aims to characterize the correlation between objective resident RCT and subjectively scored operative autonomy. Using a validated resident performance evaluation instrument, we collected resident operative autonomy ratings from residents and attendings performing robotic cholecystectomy (RC) and robotic inguinal hernia repair (IH) at a university-based general surgery program between 9/2020-6/2021. We then extracted RCT data from the Intuitive surgical system. Descriptive statistics, t-tests and ANOVA were performed. A total of 31 robotic operations (13 RC, 18 IH) performed by 4 attending surgeons and 8 residents (4 junior, 4 senior) were matched and included. 83.9% of cases were scored by both attending and resident. The average RCT per case was 35.6%(95% CI 13.0%,58.3%) for junior residents (PGY 2-3) and 59.7%(CI 51.1%,68.3%) for senior residents (PGY 4-5). The mean autonomy evaluated by residents was 3.29(CI 2.85,3.73) out of a maximum score of 5, while the mean autonomy evaluated by attendings was 4.12(CI 3.68,4.55). RCT significantly correlated with subjective evaluations of resident autonomy (r=0.61, p=0.0003). RCT also moderately correlated with resident training level (r=0.5306, p<0.0001). Neither attending robotic experience nor operation type significantly correlated with RCT or autonomy evaluation scores. Our study suggests that resident console time is a valid surrogate for resident operative autonomy in robotic cholecystectomy and inguinal hernia repair. RCT may be a valuable measure in objective assessment of residents' operative autonomy and training efficiency. Future investigation into how RCT correlates with subjective and objective autonomy metrics such as verbal guidance or distinguishing critical operative steps is needed to validate the study findings further.

Evaluating the Content Validity: Development of An Instrument for Measuring Functional Building Performance

Managing the selection of relevant criteria for functional building performance is essential for the building management team because it is strongly related with building occupants’ satisfaction and achievement of organizational objectives. In the current state of functional building performance evaluation, previous studies have shown that the criteria vary that depends on the facility types and purpose of conducting performance evaluation. These selection criteria have a direct impact on functional building performance and need to be done in in a systematic way. The content validity of the functional building performance evaluation instrument is essential step in the instrument development. Therefore, this research aims to evaluate the content validity of Functional Building Performance (FBP) evaluation by using Content Validity Index (CVI) and modified Kappa coefficient. In achieving this research aim, a set of questionnaires was developed based on numerous construct items that obtained from previous studies by various authors and researchers. The selected expert panel in the field of historic building management and building performance evaluation such as local municipality, related government agencies, academician and building management team reviewed and rated the instrument to ensure its relevance and representativeness of each item. The final instrument contained 39 items that is valid and considered to be retained and all items will be further tested in next study. The result also shown the S-CVI/ Ave for all items meet the criterion of 0.90.

Teachers’ Performance Evaluation Instrument Designs in the Implementation of the New Learning Paradigm of the Merdeka Curriculum

This research aims to design an instrument for evaluating teachers’ performance in the implementation of the new learning paradigm of the Merdeka Curriculum. This research employed a qualitative approach with a literature study method. This research data is secondary data collected through the exploration of sixty-four literary references from books and articles in nationally accredited and internationally reputable scientific journals. The data were analyzed by using Miles’ and Huberman’s interactive model, which includes data reduction, data presentation, and conclusion drawing/verification. The results of this study indicated that the design of the teacher performance evaluation instrument in the new learning paradigm of the Merdeka Curriculum take into account teachers’ planning, implementation and assessment of the learning. Lesson planning is evaluated based on the comprehensiveness of the components contained in the teaching modules composed by a teacher. The teaching module must contain indicators of general information and core components. The instrument utilized is a documentation review using a checklist sheet with a Guttman scale. The implementation of the learning is evaluated based on the teachers’ learning practices, using the indicators of the teachers’ skills in preliminary activities, core activities and closing activities. The instrument utilized in this case is an observation sheet in the form of a rubric with a 4-point Likert scale. The evaluation of the learning process is referred to as an assesssment, using the learning assessment indicators include diagnostic, formative, summative, and assessment principles and components. Teachers’ performance evaluation instruments designs in the implementation of the new learning paradigm of the Merdeka Curriculum was serve as a reference in conducting evaluations. Through performance evaluation, teachers’ competencies can be measured, learning obstacles can be identified, feedback on classroom needs can be provided, and opportunities for schools and teachers to develop themselves can be created.

Synthesis of Ambient Copolymerization CQAS Hydrogel Composite Membrane with Enhanced Antifouling

AbstractHydrogel molecular structure contained massive hydrophilic groups, which can capture water molecules to form the rejection layer of natural organic matter (NOM). Aiming at introducing the excellent properties of hydrogels into the water treatment separation membrane, a new kind of positive charged ambient self‐crosslinking chitosan quaternary ammonium salt (CQAS) hydrogel is designed and applied to prepare a hydrogel composite membrane via surface coating method. Effects of the amount of initiator on the ambient self‐cross‐linking time of CQAS hydrogels is discussed, as well as the impacts of coating amount of ambient self‐cross‐linking CQAS pre‐gel solution on the comprehensive properties of the composite membranes is also studied. The membrane structure and morphology is characterized by FT‐IR, FE‐SEM, AFM, and the membrane properties are evaluated by dynamic contact angle and membrane performance evaluation instrument. The obtained CQAS hydrogel composite membrane exhibited super hydrophilicity, high, and stable separation performance of monovalent and divalent cations, excellent long‐term tolerance, and resistance to NOM contaminants. Specifically, in the dynamic antifouling test, the stable flux recovery rate of the CQAS hydrogel membrane to the simulative contaminants BSA, HA, and SA are 82.95%, 92.89%, and 73.37%, respectively.

Multilayer-coated hydrogel membranes with comprehensive fouling resistance and stability for wastewater treatment

Organic fouling was one of the most fatal weakness for membranes applied in water treatment. Aiming at overcoming the defect of liable to contamination and achieving efficient circulation and long-term stable operation, it was reported a kind of multilayer-loaded hydrogels membranes with superior antifouling property. Hydrogel coating mixture were coated onto PES substrate layer by layer and then cross-linked by ambient free radical blending polymerization reaction endowing the membrane with interpenetrating network structure and super hydrophilic surface. The FT-IR, FESEM, and AFM were used to character the structure and morphology of obtained membranes. Dynamic contact Angle, BET and membrane performance evaluation instrument were used to evaluate the membrane performance. Compared with quantitative single layer-coated membrane, the four layers-coated hydrogel membrane exhibited more excellent and stable performance in terms of flux, rejection and fouling resistance. In particular, the water flux of the four layers-coated hydrogel membrane increased by about 25%. When 0.5 g/L BSA was selected as simulated protein contamination liquid, flux recovery rate (FRR) of four layers-coated hydrogel membranes was up to 86.33%, indicating excellent long-term tolerance and resistance to proteins. It must be pointed out that four layers were the maximum loading number of the ambient self-crosslinking hydrogel CQAS casting solution within the limit air-cured time. On the whole, the multi-layer coated hydrogel membranes showed practical application value in the field of water treatment.

Optimising Cost vs Accuracy of Decentralised Analytics in Fog Computing Environments

The exponential growth of devices and data at the edges of the Internet is rising scalability and privacy concerns on approaches based exclusively on remote cloud platforms. Data gravity, a fundamental concept in Fog Computing, points towards decentralisation of computation for data analysis, as a viable alternative to address those concerns. Decentralising AI tasks on several cooperative devices means identifying the optimal set of locations or Collection Points (CP for short) to use, in the continuum between full centralisation (i.e., all data on a single device) and full decentralisation (i.e., data on source locations). We propose an analytical framework able to find the optimal operating point in this continuum, linking the accuracy of the learning task with the corresponding network and computational cost for moving data and running the distributed training at the CPs. We show through simulations that the model accurately predicts the optimal trade-off, quite often an intermediate point between full centralisation and full decentralisation, showing also a significant cost saving w.r.t. both of them. Finally, the analytical model admits closed-form or numeric solutions, making it not only a performance evaluation instrument but also a design tool to configure a given distributed learning task optimally before its deployment.

Validity and Reliability of the Korean Version of the Holistic Nursing Competence Scale.

This methodological study aimed to verify the validity and reliability of the Korean version of the Holistic Nursing Competence Scale (HNCS), which comprises five dimensions and 36 items. The English version of the HNCS was forward and backward translated and administered to 251 participants with more than a year of work experience in a general hospital. Data were analyzed using SPSS WIN 24.0(Chicago, IL, USA), and AMOS program was used for confirmatory factor analysis. Additionally, the “Task Performance Evaluation Instrument for Clinical Nurses” was used for concurrent validity. Reliability assessed using Cronbach’s α was 0.969. Convergent, discriminant, and concurrent validity were good. Average variance extracted and construct reliability ranged from 0.845 to 0.932 and 0.980 to 0.987, respectively. The model was suitable with the chi-square value being 1216.563 (df = 584, p < 0.001), and Q value being less than three. Goodness-of-fit index, root mean square residual, and root mean square error of approximation were 0.784, 0.066, and 0.066, respectively. Moreover, comparative fit index, Tucker–Lewis index, and incremental fit index were 0.913, 0.906, and 0.913, respectively. Thus, this study verified the validity and reliability of the Korean version of the HNCS. Our findings suggest that the scale is helpful in measuring and developing the holistic nursing competence of clinical nurses.

Psychometrics of Questionnaires Designed for the Evaluation of Safety Levels and Safety Culture Case Study: Power Plant Generator Manufacturing Company

The psychometrics of instruments in safety performance evaluation is essential for the accreditation of an organization’s safety evaluation and has been emphasized in many studies. Psychometrics pertains to the validity and reliability of an evaluation instrument and describes its precision and consistency. The aim of the present study was to evaluate the psychometrics of safety level and safety culture questionnaires to provide a reliable and valid instrument for safety performance evaluation in industries. This descriptive cross sectional study was conducted with the intention of psychometric instruments used to evaluate the safety level and the safety climate at the Mapna Pars power plant generator manufacturing and engineering company. The face validity of the questionnaires was analyzed quantitatively and qualitatively. In the quantitative method, the importance of the influence score was applied and the content validity was calculated using the Lawshe method. Content Validity Ratio (CVR) and Content Validity Index (CVI) were also used. For this purpose, elicitation was obtained from experts within the professional health and safety community. The reliability of the instruments was determined via the Cronbach's alpha test and the Interclass Correlation Coefficient (ICC) test. Finally, the standardized questionnaires were used to evaluate the safety level and safety climate of the industry as a case study. The obtained data were analyzed using SPSS software solution v20. Based on the quantitative face validity results obtained at the Mapna Pars Company, one question was removed from each of the final instruments. The content reliability analysis revealed that the safety level questionnaire had a CRI of 0.99, CVR of 0.91, and was accepted. The safety climate questionnaire had an acceptable CVI of 0.95 and an acceptable CVR of 0.82. Regarding the reliability analysis, a Cronbach's alpha of 0.72 and 0.89 was obtained for the safety level questionnaire and the safety climate questionnaire, respectively. It can be concluded that both questionnaires had an acceptable level of internal consistency. The re-application of the questionnaires after two weeks revealed a relatively consistent safety level (ICC=0.90) and safety climate (ICC=0.74). Analyzing the data obtained in the present study showed that the safety level questionnaire with 66 questions and the safety climate questionnaire with 93 questions had acceptable validity and reliability. Thus, it may provide a useful approach for safety evaluations in similar industries.

A management instrument for team performance evaluation

Purpose: This article aims to develop an instrument to assess the collective performance of work teams' production. Associated with this, this evaluation instrument is applied to two production teams from the same company in Santa Catarina, to validate it. Methodology: The study is characterized as descriptive, quantitative and survey strategy. In order to develop the performance evaluation instrument, the literature review presented was taken as a starting point to guide the selection of the 10 indicators and 30 evaluation criteria enabling the elaboration f the questionnaire. Results: In general, it is possible to observe that the teams performed “good” in most of their indicators, and none of the indicators showed “poor”, “bad” or “regular” evaluations. It is suggested to develop some strategies to overcome the weakness shown by the communication indicator in both teams. Regarding the use of the instrument, it proved to be adequate to assess the collective performance, pointing out its strengths and weaknesses. Limitation: Singularity of application in a single company, not allowing generalizations to be made to other production teams. Originality: The proposal of an instrument to evaluate production teams ‘performance, indicating the team’s strengths and weaknesses.

Analysis of Human Resources Performance Evaluation Instrument at X Health Centre in Malang

X Health Centre is one of the health centres located in the Sukun sub-district of Malang, with a working area covering four villages. As a healthcare facility, X Health Centre is expected to continuously improve the quality of its services and performance. However, over a period of three years, from 2016 to 2018, the performance and the level of community satisfaction of X Health Centre has downgraded. The overall performance of the health centre depends on the total value of its employee performance. The employee performance can be determined by evaluating individual performance using performance evaluation instruments. Thus, the purpose of this study was to analyse the instrument for evaluating the performance of the employees of X Health Centre in Malang. This qualitative descriptive study used a case study design and a cross-sectional approach. The samples in this study were selected through purposive sampling techniques. Based on the results of the study, it was observed that the instrument for evaluating the performance of the employees of X Health Centre does not meet the requirements stated in the Government Regulation of the Republic of Indonesia Number 30 of 2019 about the Performance Evaluation of Civil Servants.&#x0D; Keywords: health centre, employee performance evaluation, employee performance evaluation instruments

PSYCHOMETRIC PROPERTIES OF FACULTY PERFORMANCE EVALUATION INSTRUMENT OF STATE UNIVERSITIES AND COLLEGES IN THE PHILIPPINE EASTERN VISAYAS REGION

The main concern of this study was the assessment of the faculty performance evaluation instrument of State Universities and Colleges in the Philippine Eastern Visayas Region. Specifically, the study sought answers to research questions on what are the psychometric properties of the evaluation instrument and what are the problems identified with regards to the faculty performance evaluation instrument. In search of answers to the abovementioned research questions, the researcher made use of the descriptive-assessment research design. The respondents of this study were administrators, faculty members, and students from the main campuses of State Universities and Colleges in Philippine Eastern Visayas Region. As result, the researcher found out that the faculty performance evaluation instrument of the SUCs was valid and administrable, but not yet tested for reliability. The serious problems identified on the faculty performance evaluation system of SUCs in Philippine Eastern Visayas Region were primarily related to the evaluation criteria and procedure such as the lack of involvement of administrators, faculty, and students in the construction of the evaluation system.

Pengembangan instrumen evaluasi kinerja pada Perpustakaan Universitas Kristen Satya Wacana Salatiga

Introduction. Performance evaluation is the process of evaluating the implementation of the tasks of organizational members, including libraries, in accordance with the performance standards using particular way including instruments. Instruments are needed to document the evaluation carried out by the organization. Therefore, libraries need to pay attention to developa performance evaluation instrument to plan their human resources. This research explains an overview of stages in developing performance evaluation instruments in an organization, particularly libraries.&#x0D; Research methods. The research used a descriptive qualitative approach by involving the existing performance evaluation dimensions. After that, the performance evaluation indicators were developed.&#x0D; Data analysis. The measurement of validity and reliability was valid when the correlation between items was &gt;0.3 and when Cronbach alpha was &gt; 0.6. The measurement of validity and reliability help to support the development of performance evaluation instruments.&#x0D; Results and Discussion. Of the 36 statements, only 27 items were valid and used in performance appraisal instruments. The items are arranged in a model of development of the performance evaluation instruments.&#x0D; Conclusion and recommendations. A good organization needs to pay attention to the right indicators in an assessment instrument. In its application, it is effective to use the 360 ​​degree method, where employees are evaluated by other staff to ensure the balance professional perspective.

A novel cellulose/chitosan composite nanofiltration membrane prepared with piperazine and trimesoyl chloride by interfacial polymerization.

Bamboo cellulose (BC) is one of the most abundant renewable, hydrophilic, inexpensive, and biodegradable organic materials. The cellulose membrane is one of the best materials for replacing petroleum-based polymer films used for water purification. In this study, N-methylmorpholine-N-oxide (NMMO) was used as a solvent to dissolve cellulose and chitosan, and a regenerated cellulose/chitosan membrane (BC/CSM) was prepared by phase inversion. A new kind of cellulose/chitosan nanofiltration membrane (IP-BC/CS-NFM) was obtained by the interfacial polymerization of piperazine (PIP) and trimesoyl chloride (TMC). The IP-BC/CS-NFM was characterized by Fourier transform infrared spectroscopy (FT-IR), field emission scanning electron microscopy (FE-SEM), atomic force microscopy (AFM), thermal gravimetric analysis (TGA), the retention rate, and water flux. FT-IR analysis showed that polypiperazine amide was formed. Additionally, FE-SEM and AFM showed that a uniform roughness and dense functional layer was formed on the surface of the IP-BC/CS-NFM. Furthermore, TGA analysis showed that the thermal stability of IP-BC/CS-NFM is better than that of BC/CSM. The inorganic salt retention of IP-BC/CS-NFM was measured using a membrane performance evaluation instrument, following the order R(Na2SO4) > R(MgSO4) > R(MgCl2) > R(NaCl). At a pressure of 0.5 MPa, the retention rates for NaCl, Na2SO4, MgSO4, MgCl2, Methyl Orange, and Methyl Blue were 40.26%, 71.34%, 62.55%, 53.28%, 93.65%, and 98.86%, and the water flux values were 15.64, 13.56, 14.03, 14.88, 13.28, and 12.35 L m−2 h−1, respectively. The IP-BC/CS-NFM showed better water flux and a higher rejection rate in aqueous dye-salt solutions, and had a good separation performance under different operating pressure conditions.

Level of characterization development reached by Santiago badmintonists in decision making

The act of deciding is considered one of the current research problems of Sports Psychology. It is important to know what athletes think, how they analyze game situations and how they make decisions during vital moments of obtaining a satisfactory sporting result. The decision-making guarantees, then, the rapid adaptation of the athlete to emergent situations of training and competition. Badminton as a sport of high strategy demands in those who practice it, perceptual, decisive and execution skills. From a descriptive study, having as sample six badmintonists from Santiago, it is intended to characterize the level of development reached by the latter in decision making. Techniques such as the memory stimulation interview, coach interviews and the game performance evaluation instrument (GPAI) are used as an observation protocol. The present characterization shows that in training and competition, badmintonists present difficulties in the perceptual-cognitive, decisive and executive indicators of the decision process. They also reveal the technical-methodological limitations of the coach in the development of the said process in his athletes.

A Proposal for updating the Aeronautic enlisted men’s evaluating performance instrument based on the perceptions of senior and warrant officers

Goal: identify the key competencies, in the perception of the senior officers and the warrant officers of the Air Force, necessary for the enlisted men to perform their functions and subject to consideration for updating the existing performance evaluation instrument.&#x0D; Methodology: preliminarily, a documentary and bibliographic research was carried out, capable of supporting the construction of a questionnaire, which was answered by 674 senior officers and 727 warrant officers belonging to the research target audience. Data were analyzed using centrality and dispersion parameters, also considering Pearson's coefficient of variation.&#x0D; Results: 29 competencies could be identified as being the main key competencies, highlighting those related to the alignment with military precepts.&#x0D; Limitations: it is possible to obtain a large amount of data using questionnaire, but the method does not allow to deepen the structure and processes that involve the questions asked. Also, distortions can occur caused by the subjectivity present in the respondents' perception.&#x0D; Practical implications: the results proved to be a proposal to updating the current performance evaluation system for enlisted men.&#x0D; Value: the value of this research is its contribution to a better performance evaluation system for the Brazilian Air Force enlisted men.&#x0D; Key Words: Competencies survey; Performance evaluation; Military promotion; Professional development.

Development of an Instrument for Preceptor Evaluation of Medical Graduates’ Performance: the Psychometric Properties

A preceptor has a key role in evaluating medical graduate's performance in the clinical setting. This study is conducted to develop an instrument for preceptor evaluation of medical graduates' performance in the clinical setting. A mixed-method study design, sequential exploratory approach was chosen to develop the instrument. Initial semi-structured interviews were conducted with 4 preceptors at the teaching hospitals. Five main themes emerged from the interviews. The themes were developed into a 23-item questionnaire. Nineteen Head or Assistant Head of clinical departments were asked to review the relevance of the content. The questionnaire was later sent out to 34 preceptors and 35 paramedic staffs to participate in the construct validity study by conducting exploratory factor analysis (EFA). SPSS version 21 software was used to analyze the data and Varimax rotation method was performed to simplify and describe the data structure. Review of the factor structures suggested that the most appropriate fit was 5 factors. Most of the questionnaire items were relevant to assess performance (4.65 + 0.15), except in item 4 of the clinical skill factor. The 23 items of the evaluation instruments showed that five factors were extracted which explained 73.9% of the variance between them. Construct validity was achieved after the instrument was run for an iteration of eight times, with Cronbach's alpha of 0.951. The instrument has achieved the desired content and construct validity score. It can be used by other institutions to assess their medical graduates' performance in the clinical setting.

Preparation and characterization of cellulose nanofiltration membrane through hydrolysis followed by carboxymethylation

Bamboo cellulose (BC) is hydrophilic, biodegradable and inexhaustible. The bamboo cellulose membrane (BCM) is one of the best materials to replace petroleum-based polymer film for water purification. In this study, the N-methylmorpholine-N-oxide (NMMO) was used as a solvent to dissolve cellulose 6 wt.%, and regenerated cellulose membrane was prepared by phase inversion. A new kind of cellulose nanofiltration membrane (BC-NFM) was obtained by the hydrolysis and carboxymethylation of dense cellulose membrane (BCM). The modification was carried out through hydrolysis followed by carboxymethylation. The BC-NFM was characterized by XRD, FT-IR, SEM and thermal analysis. BC-NFM performance evaluation instrument were used to evaluate retention rate and water flux of nanofiltration membrane. BCM was immersed in 1 mol/l NaOH and 3 wt/v.% chloroacetic acid solution to obtain BC-NFM. By calculating, pore size of nanofiltration membrane was 0.63 nm. With a pressure of 0.5 MPa, the water flux of nanofiltration membrane for Na2SO4 solution was 10.32 l/m2h, and the retention rate was 68.4 %. The water flux for NaCl solution was 13.12 l/m2h, and the retention rate was 34.9 %. And the retention rates were 93.0 % and 98.9 % for methyl orange and methyl blue, respectively. The stability of the nanofiltration membrane was measured by the thermal analyzer, following the order of BC>BCM>BC-NFM. The prepared cellulose nanofiltration membrane exhibited good stability in water treatment process, and can be used to remove organic compounds in aqueous solutions.

Multi-Institutional Development of a Mastoidectomy Performance Evaluation Instrument

A method for rating surgical performance of a mastoidectomy procedure that is shown to apply universally across teaching institutions has not yet been devised. This work describes the development of a rating instrument created from a multi-institutional consortium. Using a participatory design and a modified Delphi approach, a multi-institutional group of expert otologists constructed a 15-element task-based checklist for evaluating mastoidectomy performance. This instrument was further refined into a 14-element checklist focusing on the concept of safety after using it to rate a large and varied population of performances. Twelve otolaryngological surgical training programs in the United States. A total of 14 surgeons from 12 different institutions took part in the construction of the instrument. By using 14 experts from 12 different institutions and a literature review, individual metrics were identified, rated as to the level of importance and operationally defined to create a rating scale for mastoidectomy performance. Initial use of the rating scale showed modest rater agreement. The operational definitions of individual metrics were modified to emphasize "safe" as opposed to "proper" technique. A second rating instrument was developed based on this feedback. Using a consensus-building approach with multiple rounds of communication between experts is a feasible way to construct a rating instrument for mastoidectomy. Expert opinion alone using a Delphi method provides face and content validity evidence, however, this is not sufficient to develop a universally acceptable rating instrument. A continued process of development and experimentation to demonstrate evidence for reliability and validity making use of a large population of raters and performances is necessary to achieve universal acceptance.

Revisiting port performance measurement: A hybrid multi-stakeholder framework for the modelling of port performance indicators

This study develops a new port performance measurement model by taking the perspectives from different port stakeholders. The novelty lies in the modelling of interdependencies among port performance measures, and the combination of weights of interdependent measures with both qualitative and quantitative evaluations of the measures from multiple stakeholders for quantitative port performance measurement. It represents an effective performance measurement tool and offers a diagnostic instrument for performance evaluation and/or monitoring of ports and terminals so as to satisfy different requirements of various port stakeholders in a flexible manner.