Performance Evaluation Framework Research Articles

Holistic performance evaluation framework: power distribution network health index

In 2015, China National Energy Administration announced an action plan of 2 trillion yuan ($3225.8 billion) investment for power distribution network for the coming 5 years. The intent is to upgrade and modernise the network to meet multidimensional requirements, including distributed energy resources, high levels of security, quality and efficiency of power supply, and requests of CO 2 reduction. A systematic, effective reform action would best be based on the comprehensive evaluation of network performance. However, in literature, previous works evaluated intensively on partial network performances, i.e. reliability, operation efficiency etc. Different from these approaches, this study proposes a holistic performance evaluation framework based on health index (HI) methodology. To cope fully with industry needs, the proposed HI system measures performances of two-level objects from power equipment to network scales. For each level, the corresponding HI is derived successively by determining object's health metrics, establishing a core set of indicators, building physics-based model, using data-driven and multiple health information fusion techniques. As part of a 3-year major research programme initiated by State Grid of China, the above work has been implemented in a software architecture. The fundamentals of distribution network HI framework are reported and discussed.

Energy/Quality-of-Experience TradeOff of Power Saving Modes for Voice Over IP Services

Deploying a power saving mode (PSM) is considered an effective solution for saving energy; however, a key challenge is how to simultaneously save energy and support a required level of quality of experience (QoE) using PSM for voice over IP (VoIP) services. This paper aims at developing a novel Markovian-based evaluation framework that assesses the energy consumption and QoE, achieved by a fixed sleeping time-based PSM. Based on this framework, a novel QoE-aware PSM is developed, which exploits the ON-OFF characteristics of a VoIP session to further improve the energy saving, while accounting for the effect of bursty losses on the QoE. The QoE and energy consumption achieved by the proposed PSM are evaluated via solving a system of linear equations. This facilitates the analytical formulation of the QoE/energy tradeoff that is needed for the successful integration of this QoE-aware PSM within the traffic offloading mechanisms. Numerical results validate the accuracy of the performance evaluation framework and verify the efficacy of the proposed QoE-aware PSM in saving energy while assuring a desired level of QoE. The Pareto-optimal operational regime show that the proposed QoE-aware PSM significantly reduces power consumption at a desired level of QoE compared with its counterparts.

Energy-Based Seismic Risk Evaluation of Tall Reinforced Concrete Building in Vancouver, BC, Canada, under Mw9 Megathrust Subduction Earthquakes and Aftershocks

This paper presents a seismic performance evaluation framework for reinforced concrete (RC) buildings, comprising of shear-walls and gravity frames. The evaluation is undertaken within a performance-based earthquake engineering framework by considering regional seismicity and site-specific ground motion selection. Different engineering demand parameters (EDPs), i.e. maximum inter-story drift ratio and energy-based damage index, are considered as performance indicators. Various prediction models of EDPs are developed by considering four ground motion intensity measures (IMs), i.e. spectral acceleration at the fundamental period, Arias intensity, cumulative absolute velocity, and significant duration of ground motion. For this study, a 15-story RC building, located in Vancouver, British Colombia, Canada, is considered as a case study. Using 50 mainshock and 50 mainshock-aftershock earthquake records (two horizontal components per record and bi-directional loading), nonlinear dynamic analyses are performed. Subsequently, the calculated maximum inter-story drift ratios and damage indices are correlated with suitable IMs using cloud analysis, and the most efficient IM-EDP prediction models are selected by comparing standard deviations of the regression errors. The maximum inter-story drift ratio of the shear-walls is less than 1% for the mainshock and mainshock-aftershock records. The energy-based damage index shows sensitivity to delineate impact of earthquake types and aftershocks. The cumulative absolute velocity is showed to be the most efficient IM for the energy-based damage index.

Sensitivity analysis of seismic performance assessment and consequent impacts on loss analysis

The probabilistic framework for seismic performance evaluation developed by PEER has been widely used in the literature. This framework consists of four steps: hazard analysis, response analysis, damage analysis, and loss analysis. Typically the process involves ground motion selection for numerical analysis, probabilistic model development for engineering demand parameters (EDP), and an EDP hazard calculation approach. This study investigates the impact of three aspects (i.e., the ground motion suite selected, the EDP formulation chosen, and the hazard calculation formulation adopted) on the seismic performance of an office building located in downtown Los Angeles. The result of this study shows that these three aspects have significant impacts on the seismic performance in terms of dynamic responses, demand hazard, and expected annual loss. This study also suggests that more accurate and robust performance evaluation is obtained when: (1) using a ground motion suite that contains a rich intensity measure content, (2) using vector-valued demand models that provides more accurate predictions rather than scalar-valued demand models, and (3) using joint hazard formulations when two seismic intensity measures are involved.

Partner selection and performance evaluation framework for a construction-related virtual enterprise: a multi-agent systems approach

ABSTRACTTo exploit market opportunities, disparate enterprises can pool their core competencies to form a temporary organisation. This inter-organisational collaboration of enterprises is commonly referred to as a virtual enterprise (VE). However, the VEs’ partner selection problems influence their overall performance. Due to the limited construction-specific studies on the evaluation and selection of partners, the current research sought to address this knowledge gap by investigating the partner selection problem. The partners comprising 10 construction companies carrying out project tasks for a large building in Nairobi, Kenya, were evaluated. Data were collected using a sequential mixed-methods approach consisting of a focus group interview, questionnaires and algorithms. The study proposed the use of the partner selection and performance evaluation technique. This technique combines fuzzy approximate reasoning with the conventional analytic hierarchy process algorithm, designed to deal with imprecise evaluators’ judgments. A multi-agent systems approach was chosen to simulate VEs. Results show that the chosen techniques are both efficient and effective. In particular, the reduced group fuzzy analytic hierarchy process reduces the number of pairwise comparisons required when comparing a large number of attributes. Application to the construction management domain, in particular, is germane to the research.

A simulation framework for network performance evaluation of large-scale RF-mesh AMIs

Advanced Metering Infrastructures (AMIs) lay the foundation of a wide variety of smart-grid applications. Despite their low capacity, RF-mesh systems are an increasingly popular option for AMI implementation. In this paper, we present a simulation framework for realistic RF-mesh AMIs with the objective of analyzing the performance of this kind of system, and of defining the feasibility of possible smart-grid applications. The framework contains a simulation tool, implemented in Java and Python, that permits the simulation of large-scale AMIs, while taking into account important technical details in a reasonable amount of time. Numerical results obtained in a real case scenario are also proposed and discussed.

Performance assessment of human resource by integration of HSE and ergonomics and EFQM management system

PurposeThe purpose of this paper is to present an integrated framework for performance evaluation and analysis of human resource (HR) with respect to the factors of health, safety, environment and ergonomics (HSEE) management system, and also the criteria of European federation for quality management (EFQM) as one of the well-known business excellence models.Design/methodology/approachIn this study, an intelligent algorithm based on adaptive neuro-fuzzy inference system (ANFIS) along with fuzzy data envelopment analysis (FDEA) are developed and employed to assess the performance of the company. Furthermore, the impact of the factors on the company’s performance as well as their strengths and weaknesses are identified by conducting a sensitivity analysis on the results. Similarly, a design of experiment is performed to prioritize the factors in the order of importance.FindingsThe results show that EFQM model has a far greater impact upon the company’s performance than HSEE management system. According to the obtained results, it can be argued that integration of HSEE and EFQM leads to the performance improvement in the company.Practical implicationsIn current study, the required data for executing the proposed framework are collected via valid questionnaires which are filled in by the staff of an aviation industry located in Tehran, Iran.Originality/valueManaging HR performance results in improving usability, maintainability and reliability and finally in a significant reduction in the commercial aviation accident rate. Also, study of factors affecting HR performance authorities participate in developing systems in order to help operators better manage human error. This paper for the first time presents an intelligent framework based on ANFIS, FDEA and statistical tests for HR performance assessment and analysis with the ability of handling uncertainty and vagueness existing in real world environment.

Performance evaluation of outsourcing decision using a BSC and Fuzzy AHP approach: A case of the Indian coal mining organization

Mineral and mining sectors are always of a great concern to any nation due to its major contribution to the economy. In India, the demand for coal is continuously on rise due to its ever increasing need from the growing power sectors and steel industries. In spite of the large coal reserves, India has to import coal from overseas sources to address the perpetual demand-supply gap. In order to reduce the dependence on imported coal, to ensure an affordable price to the domestic customers as well as to achieve operational efficiency, the state-owned coal mining organization of India have now started taking initiatives to outsource some of the operational activities involving private agencies. To realize the success of outsourcing, it is indispensable to consider it as part of the corporate decision. Such decision essentially considers all possible attributes of strategy planning for performance improvement. The study focuses on the development of an effective performance evaluation framework based on Balanced Scorecard (BSC) and Fuzzy Analytic Hierarchy Process (FAHP) to analyze the suitability of organization's strategic decision of outsourcing in alignment with the organizational performance for the Indian coal mining organization. BSC administers strategic elements of decision making in assessing the performance of the firm whereas FAHP, on the other hand, is applied to determine the relative importance weight of criteria in regard to organizational objectives taking into consideration the vagueness and ambiguity of information as characteristics of decision-making problems. The findings of the present study establish the proposed framework as an analytical tool in strategy formulation and provide rationale guidance to management with regard to performance improvement.

Policy-aware assessment of environmental impacts from transport in smart cities

In recent research a performance evaluation framework for traffic management and Intelligent Transport Systems was developed, consisting of a set of Key Performance Indicators (KPIs) for the themes of traffic efficiency, safety, pollution reduction and social inclusion, all of which are key components of a smart city. One of the innovative elements of these KPIs is their ability to consider the transport policy layer, in the sense that the evaluation of the suitability and effectiveness of different strategies and ITS options is calculated in relation to the decision-maker’s high-level transport policy rather than objectively. This is achieved through weighting factors, whereby more important policy objectives are weighted more heavily in the calculation. But while the theoretical framework is ready to accommodate the policy layer, no methodology to determine the values of the weighting factors has been developed so far. The present study, therefore, concentrates on the development and testing of such a methodology, focusing on the environmental impact aspect of urban mobility management and ITS in the context of smart cities. The development is based on existing policy objectives and legislation in different cities and countries, while testing is carried out using the purpose-developed CONDUITS_DST software with data from microsimulation models before and after the implementation of a bus priority signalling system in Brussels, Belgium. The results show that the method captures the expected effects, but also that it is able to reflect policy objectives and deliver evaluation results in relation to their alignment with those

Modeling and performance evaluation of agility coupled with sustainability for business planning

PurposeAgility and sustainability are different manufacturing paradigms. Ideally, companies should merge the two concepts so as to enhance competitive advantage. Agility, which has evolved from lean manufacturing, is the ability of an enterprise to respond quickly to a wide range of customer demands. On the other hand sustainability is a condition in which a company optimizes economic, social and environmental factors by taking into account indirect and long-term consequences. Merging the two concepts could result in holistically improving the system, by fulfilling performance without depleting available resources. Thus, identifying and simultaneously improving two areas with common goals could lead to developing a successful and an emerging enterprise. The paper aims to discuss these issues.Design/methodology/approachThe authors identify agility-sustainability criteria which complement each other from amongst a set of differentiating criteria. The conceptual model is built by integrating five enablers with 25 criteria and 75 attributes. The combined criteria are modeled using graph theory which considers interdependencies and contributing factors on the organization. Within each enabler, the authors investigate the interdependencies between criteria and focus further on those that have potential to improve overall score. The methodology was applied in a manufacturing organization that operates on ad hoc basis depending on projects’ availability.FindingsThe model determines a range of values for extreme scenarios under theoretical and practical situations, along with the current level of business. The scenarios are tested for assessing performance by identifying the gaps in the weaker areas.Research limitations/implicationsApproach can serve as a managerial tool for monitoring performance of the organization along agility and sustainability dimensions. Further, government entities can use the approach in devising policy for improving economies by considering holistic approach rather than only environmental aspect. Biased and conflicting opinions of experts might deter chances for comprehensive and consistent decision.Originality/valueThe study integrates a framework for modeling and performance evaluation for a system that incorporates agility and sustainability characteristics. The study will enable practicing managers and decision makers to identify suitable characteristics that would simultaneously improve an organization’s agile performance by considering aspects of sustainability.

China’s megaregion policy: Performance evaluation framework, empirical findings and implications for spatial polycentric governance

Megaregion has emerged as a new dimension of global urbanization. A megaregion approach based on polycentric strategy is deemed to enhance regional economic competitiveness. Numerous studies have highlighted the economic benefits, celebrated the spill-over effects, and outlined the creative potentials of contemporary megaregion policies in different nations worldwide. However, further policy instruments require the knowledge about the achievement and failure of megaregion policies that seek for spatial, economic, social, and environmental efficiency and sustainability. This paper introduces China’s megaregion policy and proposes an analytical framework for performance evaluation from four principle domains (rational urban growth, economic development, social equity, environmental protection) at three levels (internal collaboration, integral development, and overall development). Using a case of the Megaregion around Hangzhou Bay (MAHB), we find very limited success of China’s megaregion policy. In particular, the megaregion policy only accomplishes the economic goal, and fails to achieve the goals of rational urban growth, environmental protection, and social equity. A series of mechanism based regressions are established and show that the implementation duration of megaregion policy: (1) associates positively with the economic growth; (2) relates negatively to social equity and urban rational growth; and (3) has no significant relationship with improved environmental quality. Institutional fragmentation, no unified spatial planning, and inadequate legislation at megaregional level are the underlying causes of the expected performances within the policy context of governance itself. We finally propose some possible solutions and discuss the implications for spatial polycentric governance. The demonstrated methodological framework can be applicable to other megaregions around the world. This paper is thus believed to provide some new insights for land use policy.

Sustainable Manufacturing Performance Evaluation: Integrating Product and Process Metrics for Systems Level Assessment

This paper builds on the previously developed product sustainability index (ProdSI) and process sustainability index (ProcSI), and presents a framework for sustainable manufacturing performance evaluation at the systems level. The framework is then used to propose a comprehensive set of metrics for the enterprise level following a five-stage metrics hierarchy (individual metrics, sub-clusters, clusters, sub-index and index). The 6R concept (reduce, reuse, recycle, recover, redesign and remanufacture), total life-cycle emphasis, and triple bottom line (TBL) are considered for selecting relevant metrics. Finally, the metrics are integrated to develop an index for enterprise level sustainability assessment and demonstrated using a numerical example.

Multi-Objective Metrics to Evaluate Deduplication Approaches

Data deduplication is a lossless compression technology that has been widely used in storage systems for space optimization. However, due to the removal of redundant data, the data deduplication has negative influences on data writing, data reading, and data reliability. In this paper, we propose a multi-objective-based performance evaluation framework to analyze the data deduplication performances and evaluate existing well-known deduplication approaches with multiple performance objectives, including compression ratio, data read performance, data write performance, and data reliability. Based on the proposed multi-objective framework and the obtained evaluation results, we further propose a multi-objective-based optimization method. Though our extensive experimental evaluation driven by real-world data sets, it is shown that this method can improve data read/write performance and data reliability while at the cost of little compression ratio.

Management Practice of Supply Chain Quality Management in Service-oriented Manufacturing Industry

Supply chain quality management (SCQM) in service-oriented manufacturing industries is needed in delivering value to customers in all process of supply chain management based on total quality management (TQM). In the paper, we identify the latest themes through reviewing prior quality management and supply chain management(SCM) literature. In particular, we find manufacturing firms transform from providing products towards providing services, which means they need to go through fundamental changes especially in supply chains. We use a case study of Heilan Home to illustrate the SCQM themes and their ways in industrial practice. Based on our research, the case study, the experience of working with this firm, we propose a SCQM performance evaluation framework, as well as four strategies for other industries to improve customer satisfaction and added value. In this way can promote the transformation and upgrading of the manufacturing industry in the perspective of quality.

An integrated framework for evaluating the performance of age progression algorithms

Facial age progression can play an important role in biometric authentication as it enables the long-term person identification based on age progressed facial renderings. In this paper, a systematic evaluation approach that can be used for assessing the performance of age progression algorithms is presented. The proposed method relies on the use of a dedicated dataset in conjunction with the development and use of machine-based performance evaluation metrics that allow the assessment of the intensity of aging effects added on a face along with an assessment of the identity preservation in age progressed images. The proposed performance evaluation framework can form the basis of implementing comprehensive comparative performance evaluation between different age progression methodologies, allowing in that way the evolution of the best algorithms.

Log-Normal Approximation for VDSL Performance Evaluation

A performance analysis of VDSL2 technologies in FttC or FttC/FttDp hybrid scenarios requires knowledge of the access network geometry, propagation, and crosstalk interference models. A simulation-based approach is commonly adopted for performance assessment. In this paper, we introduce a performance evaluation framework based on the log-normal approximation for the probabilistic description of the signal-to-interference plus noise ratio (SINR), which is then used to obtain the statistics of the achievable bit rate per user. A very good agreement has been observed between the exact and the approximated calculations of the bit rate up to 600 m from the Cabinet. In addition, the log-normal approximation has allowed to define a simple procedure, based on the equivalent number of co-located interferers , to evaluate the bit rate per user for generic access network geometry. Furthermore, two approximated formulas for bit rate have been obtained under high SINR and dominant far-end crosstalk (FEXT) conditions. They evidence the logarithmic decrease of the bit rate with the distance from the Cabinet and provide the accurate results for distances up to 400 m. Finally, when FEXT fluctuations are modeled in accordance with Beta distribution, we have observed the log-normal approximation still provides valid probabilistic description of the SINR.

Establishment and application effects of nursing personal performance appraisal program

Objective To explore and build a real, objective, comprehensive clinical nursing individual performance indicators architecture model, and check the rationality and validity for prize distribution by clinical application. Methods The methods included that discuss new ideas of nursing performance management by multi-disciplinary experts, developed clinical personal nursing staff performance evaluation program, worded out indicators and methods for the clinical assessment of individual nurses and nurse managers respectively, then applied research in the pilot departments and hospital step by step. Results A personal performance evaluation framework model was constructed, which include clinical nurses and nursing managers. Experimental results show that the nursing staff in this regard performance program have a high degree of recognition, 98.82% (1 741/1 762) nursing staff understanding of the purpose and significance, 97.15% (1 712/1 762) nurses think the performance model structure is reasonable. After the implementation of the performance program, the outstanding rate of personal performance appraisal of nurses was 93% (1 639/1 762). Conclusions The application of scientific performance appraisal programs can play a positive role in helping improve the quality of clinical care, and promote the stable development of the care team. Key words: Nursing care; Personal management; Performance evaluation

FS-IIoTSim: a flexible and scalable simulation framework for performance evaluation of industrial Internet of things systems

This paper presents a flexible and scalable network simulation framework (FS-IIoTSim) for performance evaluation of the Industrial Internet of Things (IIoT) system. FS-IIoTSim provides flexibility and scalability for performance analysis of the IIoT system by supporting not only existing legacy industrial sensor network communication models, but also user protocol modification, in order to accommodate the complexity of systems such as communication standards, smart machine traffic, and applications over the complex IIoT environment. The FS-IIoTSim consists of three parts: (1) scenario modeling, (2) performance evaluation, and (3) user interface. Each of these parts interacts and cooperates with each other to simulate a target IIoT system. Scenario modeling utilizes user simulation scenarios to create an industrial sensor network (ISN) system model and perform simulations to generate a trace file recorded with log information about the device’s operations. Performance evaluation is an analytical process of a trace file, generated in the scenario modeling part, which is used to derive a user’s preferred performance metric. The user interface part plays a role in interfacing with the detailed operations of the other parts. Within the user interface part, the user can set and control the FS-IIoTSim framework. In order to prove the effectiveness of this FS-IIoTSim, experiments were conducted in various industrial application scenarios, and simulation performance was compared and analyzed with a dedicated simulator. The experimental results showed that FS-IIoTSim provided more effective approaches to modeling and performance analysis of real ISN environments, compared to a dedicated simulator.

Performance Analysis of Correlated Handover Service in LEO Mobile Satellite Systems

Analytical models for low earth orbit mobile satellite systems (LEO-MSSs) should consider service time correlation in adjacent satellite spot beams. This letter presents a simple analytical framework for the efficient performance evaluation of new and handover services in LEO-MSSs with correlated service times. When the fading channel is temporally correlated, channel characteristics in the preceding and succeeding spot beams do not change dramatically, especially for services with short transmission times. A closed-form solution of the correlated queue service time is derived using probability generating functions whose moments are used to derive the blocking probabilities of handover and new services. The system is then modeled as a queuing system with correlated service time and its performance investigated.

The Performance Evaluation Framework for Cultivating the Technology Transfer

The concept of technology transfer has been changed for successful technology transfer with rapid technology innovation. There is a need to develop the performance evaluation framework for cultivating the technology transfer considering the change of definition of technology transfer. The suggested framework involves the phase of redefining the technology transfer as a shared two-way process, the phase of expanding the value chain, the phase of restructuring technology transfer barriers and the phase of deriving the hierarchy structure for evaluating the technology transfer containing strategies and critical success factors. Based on the suggested framework, officers in technology transfer would extract the important information and implementation plan for successful technology transfer based on strategies and critical success factors. The technology transfer of public research institutions would be cultivated by evaluating the performance of technology transfer through the proposed framework.