Performance Evaluation Methodology Research Articles

MARCOS technique under intuitionistic fuzzy environment for determining the COVID-19 pandemic performance of insurance companies in terms of healthcare services

Assessing and ranking private health insurance companies provides insurance agencies, insurance customers, and authorities with a reliable instrument for the insurance decision-making process. Moreover, because the world’s insurance sector suffers from a gap of evaluation of private health insurance companies during the COVID-19 outbreak, the need for a reliable, useful, and comprehensive decision tool is obvious. Accordingly, this article aims to identify insurance companies’ priority ranking in terms of healthcare services in Turkey during the COVID-19 outbreak through a multi-criteria performance evaluation methodology. Herein, alternatives are evaluated and then ranked as per 7 criteria and assessments of 5 experts. Experts’ judgments and assessments are full of uncertainties. We propose a Measurement of Alternatives and Ranking according to the Compromise Solution (MARCOS) technique under an intuitionistic fuzzy environment to rank insurance companies. The outcomes yielded ten insurance companies ranking in terms of healthcare services in the era of COVID-19. The payback period, premium price, and network are determined as the most crucial factors. Finally, a comprehensive sensitivity analysis is performed to verify the proposed methodology’s stability and effectiveness. The introduced approach met the insurance assessment problem during the COVID-19 pandemic very satisfactory manner based on sensitivity analysis findings.

A novel approach towards evaluation of joint technology performances of battery energy storage system in a fuzzy environment

The joint technological features of battery energy storage systems (BESSs), which are a strategic reserve resource with different operating systems, make it possible for these technologies to be comparable with each other. However, technology needs to be evaluated with stakeholders in the energy system to eliminate some engineering and feasibility concerns, to be included in energy storage policies, and to be utilizable. This aim has involved a high level of complexity and uncertainty, as well as many qualitative and quantitative criteria in both technology and stakeholder assessment. Therefore, in the study, Pythagorean Fuzzy (PF) sets, which offer the decision-maker the advantage of a wide evaluation scale, were used with AHP-TOPSIS combination, which has proven effective in many different applications in the literature. The first use of this methodology for joint technology performance evaluation of BESSs will be a visionary contribution to the literature. In the case study, 5 different BESSs (Lead-acid, Lithium-ion, Vanadium redox flow battery, Sodium nickel chloride, Sodium-sulfur) alternatives have been evaluated with the joint technological criteria of these alternatives. The results show that the most important criteria in battery performance are lifetime calendric, energy rating/discharge time, and life cycle criteria, respectively. Another conclusion of the case study is that while the rapidly developing Lithium-ion battery takes the first place in future projections, it takes second place in Lead-acid batteries, which are mature technology.

DEVS-based experimental framework for blockchain services

Blockchain works according to the principles of distributed transactions and processing tiny hash-based blocks. Data located in the blockchain preserves its integrity and transparency, which enables participants to verify it. Today, people apply blockchain technology in domains that must maintain the integrity of the data. However, people can be reluctant to introduce blockchain technology due to the complexity of the technology and the resultant inefficiency. This work presents a DEVS-based experimental methodology for performance evaluation of blockchain-based services. The experimental methodology is tested by implementing a content delivery service by utilizing the Ethereum blockchain framework. As the evaluation result, we show the overall turnaround time of the service and investigate the delay of smart contract function consisting the service. In addition, the evaluation result shows the difference in performance between blockchain-based and database-based services. This work presents a methodology for evaluating blockchain service performance and shows experimental results that can be referenced by people who are planning to implement blockchain-based services.

Ambiguity of objective image quality metrics: A new methodology for performance evaluation

Objective image quality metrics try to estimate the perceptual quality of the given image by considering the characteristics of the human visual system. However, it is possible that the metrics produce different quality scores even for two images that are perceptually indistinguishable by human viewers, which have not been considered in the existing studies related to objective quality assessment. In this paper, we address the issue of ambiguity of objective image quality assessment. We propose an approach to obtain an ambiguity interval of an objective metric, within which the quality score difference is not perceptually significant. In particular, we use the visual difference predictor, which can consider viewing conditions that are important for visual quality perception. In order to demonstrate the usefulness of the proposed approach, we conduct experiments with 33 state-of-the-art image quality metrics in the viewpoint of their accuracy and ambiguity for three image quality databases. The results show that the ambiguity intervals can be applied as an additional figure of merit when conventional performance measurement does not determine superiority between the metrics. The effect of the viewing distance on the ambiguity interval is also shown.

Applicability of statistical analysis for performance and reliability evaluation of large-scale water treatment plants with direct filtration systems.

Water treatment plants (WTPs) are extremely important in basic sanitation services because of their association with human health by producing safe drinking water. Thus, their proper operation is of utmost relevance and has led to the development of distinct performance evaluation methodologies. Direct filtration is a leading technology applied in WTPs. However, although it costs less than conventional treatment, it might also be less flexible and robust. To evaluate performance with data from real-scale WTPs with direct filtration systems, the use of a statistical analysis methodology for turbidity, apparent color, and pH data from raw and effluent water is proposed. Reliability analysis, a probabilistic-based methodology, was applied for turbidity alongside evaluating the compliance of the treated water with different potability standards. The parameters pH (between 6.0 and 8.0) and apparent color (< 15 HU) showed almost complete compliance (> 99%). Reliability analysis could not be applied for apparent color and pH, while apparent color did not adhere to the lognormal distribution frequency. The turbidity results show high variability in the coefficients of variation and reliability among various plants and a general difficulty in complying with stricter standards, such as the 0.1 and 0.3 NTU defined by the United States Environmental Protection Agency. All WTP studies show a 95% compliance with the World Health Organization standard of 5.0 NTU. A higher value for the coefficient of reliability and a lower value for the variation coefficient could indicate a more stable process, regardless of the treated water quality, highlighting the relevance of applying combined methods for performance evaluation, such as compliance with established standards.

Virtual Network Resource Optimization Model for Network Function Virtualization

Network function virtualization (NFV) is a concept aimed at achieving telecom grade cloud ecosystem for new‐generation networks focusing on capital and operational expenditure (CAPEX and OPEX) savings. This study introduces empirical throughput prediction model for the virtual network function (VNF) and network function virtualization infrastructure (NFVI) architectures based on Linux kernel. The model arises from the methodology for performance evaluation and modeling based on execution area (EA) distribution by CPU core pinning. EA is defined as a software execution unit that can run isolated on a compute resource (CPU core). EAs are derived from the elements and packet processing principles in NFVIs and VNFs based on Linux kernel. Performing measurements and observing linearity of the measured results open the possibility to apply model calibration technique to achieve general VNF and NFVI architecture model with performance prediction and environment setup optimization. The modeling parameters are derived from the cumulative packet processing cost obtained by measurements for collocated EAs on the CPU core hosting the bottleneck EA. The VNF and NFVI architecture model with performance prediction is successfully validated against the measurement results obtained in emulated environment and used to predict optimal system configurations and maximal throughput results for different CPUs.

Performance Evaluation of Containerization in Edge-Cloud Computing Stacks for Industrial Applications: A Client Perspective

Today, the edge-cloud computing paradigm starts to gain increasing popularity, aiming to enable short latency, fast decision-making and intelligence at the network edge, especially for industrial applications. The container-based virtualization technology has been put on the roadmap by the industry to implement edge-cloud computing infrastructures. Has the performance of the container-based edge-cloud computing stacks reached industry requirement? In this paper, from the industrial client perspective, we provide a performance evaluation methodology and apply it to the state-of-the-art containerization-based edge-cloud computing infrastructures. The influences of the message sending interval, payload, network bandwidth and concurrent devices on full stack latency are measured, and the processing capability of executing machine learning tasks are benchmarked. The results show that containerization on the edge does not introduce noticeable performance degradation in terms of communication, computing and intelligence capabilities, making it a promising technology for the edge-cloud computing paradigm. However, there is a large room for performance improvement between current implementation of the edge-cloud infrastructure and the demanding requirements anticipated by time-critical industrial applications. We also emphasize and showcase that partitioning of an industrial application into microservices throughout the whole stack can be considered during solution design. The proposed evaluation methodology can be a reference to users of edge-cloud computing as well as developers to get a client perspective overview of system performance.

BHA-Design Approach Improves Drilling Performance and Wellbore Quality

This article, written by JPT Technology Editor Judy Feder, contains highlights of paper SPE 199617, “Drilling Performance Evaluation Using Advanced BHA Modeling and Field Validation,” by Jeremy Greenwood, SPE, Julien Marck, SPE, and Vakkeyil Naveen Nair, Halliburton, et al., prepared for the 2020 IADC/SPE International Drilling Conference and Exhibition, Galveston, Texas, 3-5 March. The paper has not been peer reviewed. Implementing a physics-based digital twin of a drilling system can enable the drilling team to leverage data at each stage of the engineering process to deliver more-consistent, repeatable drilling performance and improved borehole quality, which in turn enables drilling farther and faster while increasing downhole tool life. The complete paper discusses a new performance-evaluation methodology that combines bottomhole assembly (BHA) modeling with field data. BHA modeling simulates the drilling process accurately to establish key performance indicators (KPIs) to help optimize BHA designs to deliver improvements in drilling performance and wellbore quality. The model also can estimate quantities such as microtortuosity that are not directly measured by standard equipment. Importance of Effective BHA Performance Evaluation Determining the cumulative effect of BHA behavior during drilling on the quality of the wellbore and the subsequent impact on performance and life of the BHA is an important goal for improving overall drilling and well-delivery efficiencies. A higher-quality wellbore not only has a positive effect on reducing the time required for all drilling, casing, or completions running operations; in curved holes, it can also extend the life of the drilling equipment by reducing the rate of accumulation of fatigue and reducing the amount of work the bit is required to perform. This leads to longer runs because the system is more mechanically efficient and has a reduced equipment failure rate. Developing an effective set of measurements allows for an objective comparison of different BHA designs and stabilization, bit selections, and downhole drive systems of rotary steerable tools, motors, or turbines. This quantification of BHA performance can be used to improve BHA designs in terms of component stiffness, amount and position of stabilization, type and configuration of the drive system, and bit-design features. Defining terms to describe borehole quality at different scales is necessary to develop indices that are comparable across wells and that reflect any deviation of the wellbore from a perfectly smooth path. To fully quantify the mechanical effect of wellbore quality on the BHA, a wellbore-propagation model was developed to replicate the borehole and determine the accumulated stresses as the BHA passes through it. This approach overcomes the limitations of not having the required resolution of downhole measurements that accurately describe the wellbore trajectory on the smaller scale. The validation of the propagation model has been completed on multiple runs with examples presented in the complete paper.

An UML Based Performance Evaluation of Real-Time Systems Using Timed Petri Net

Performance is a critical non-functional parameter for real-time systems and performance analysis is an important task making it more challenging for complex real-time systems. Mostly performance analysis is performed after the system development but an early stage analysis and validation of performance using system models can improve the system quality. In this paper, we present an early stage automated performance evaluation methodology to analyse system performance using the UML sequence diagram model annotated with modeling and analysis of real-time and embedded systems (MARTE) profile. MARTE offers a performance domain sub-profile that is used for representing real-time system properties essential for performance evaluation. In this paper, a transformation technique and transformation rules are proposed to map the UML sequence diagram model into a Generalized Stochastic Timed Petri net model. All the transformation rules are implemented using a metamodel based approach and Atlas Transformation Language (ATL). A case study from the manufacturing domain a Kanban system is used for validating the proposed technique.

An integrated TOPSIS-MOORA-based performance evaluation methodology for the key service providers in sharing economy: case of Airbnb superhosts

PurposeIn the context of sharing economy, the superhost program of Airbnb emerges as a phenomenal success story that has transformed the tourism industry and garnered humongous popularity. Proper performance evaluation and classification of the superhosts are crucial to incentivize superhosts to maintain higher service quality. The main objective of this paper is to design an integrated multicriteria decision-making (MCDM) method-based performance evaluation and classification framework for the superhosts of Airbnb and to study the variation in various contextual factors such as price, number of listings and cancelation policy across the superhosts.Design/methodology/approachThis work considers three weighting techniques, mean, entropy and CRITIC-based methods to determine the weights of factors. For each of the weighting techniques, an integrated TOPSIS-MOORA-based performance evaluation method and classification framework have been developed. The proposed methodology has been applied for the performance evaluation of the superhosts (7,308) of New York City using real data from Airbnb.FindingsFrom the perspective of performance evaluation, the importance of devising an integrated methodology instead of adopting a single approach has been highlighted using a nonparametric Wilcoxon signed-rank test. As per the context-specific findings, it has been observed that the price and the number of listings are the highest for the superhosts in the topmost category.Practical implicationsThe proposed methodology facilitates the design of a leaderboard to motivate service providers to perform better. Also, it can be applicable in other accommodation-sharing economy platforms and ride-sharing platforms.Originality/valueThis is the first work that proposes a performance evaluation and classification framework for the service providers of the sharing economy in the context of tourism industry.

Co-Heating method for thermal performance evaluation of closed refrigerated display cabinets

In this study, an application of the adapted Co-Heating methodology for thermal performance evaluation of closed refrigerated display cabinets (RDCs) has been presented. A novel test series comprising three experiments has been developed and demonstrated on a commercial RDC with four doors to evaluate the envelope heat transfer coefficient, thermal inertia, infiltration at idle state and dynamic infiltration caused by door operations. The latter two experiments were conducted in parallel with the condensate collection method for validation of the results for infiltration. It was concluded with good (<10%) conformance between the methods that the infiltration at idle state for the tested RDC is approximately 0.022 kg/s and that one 15 s door opening causes approximately 0.94 kg of ambient indoor air to infiltrate. Additionally, the time, equipment and associated costs for running the tests were compared, and it was concluded that the adapted Co-Heating methodology could substitute the condensate collection method for the evaluation of infiltration while providing additional results on the thermal performance.

Rotor Design Optimization of Squirrel Cage Induction Motor - Part I: Problem Statement

Squirrel cage induction motor is the most widely adopted electrical machine in applications directly fed by the main grid. The analysis, design, and optimization of this machine topology has been addressed by a considerable amount of literature over the last century. Although its wide adoption, the induction motor design, especially when carried out in an automatic fashion, still presents significant challenges because the accurate prediction of the performance requires time-consuming finite element analysis. This work proposes a systematic approach to perform the design optimization of a squirrel cage induction motor focusing on the rotor slot geometry, being this the major player in defining the torque-speed characteristic. Structured as a two-parts companion papers, this first part presents an innovative performance evaluation methodology which allows a very fast estimation of the torque, and efficiency behaviour preserving the results’ accuracy. The proposed performance estimation technique is assessed against experimental tests carried out on an off-the-shelf induction motor. The selection of the performance indexes to be optimized is justified in detail along with the description of a generalized rotor parametrization which allows a comprehensive exploration of the research space. The presented optimization procedure is then applied to a case study, and the preliminary results are commented, highlighting benefits, and drawbacks of the proposed methodology.

Propellant Actuated Device for Parachute Deployment during Seat Ejection for an Aircraft Application

Propellant Actuated Devices (PAD) are installed on various combat aircraft of Air Force and Naval bases to perform extremely important operations such as parachute deployment, harness and leg restrain, cable cutting, pullers, seat ejection, bomb release or fuel tanks etc. They are basically called as gas generators. Such devices produce the high temperature and pressure combustion gases on initiation and used to perform different operations. These cartridges are single-shot operating devices. The performance of such kind of PADs cannot be tested by non-destructive techniques. Hence, cartridges are designed to function with high reliability and stringent quality control checks at all levels during entire development cycle. The safety features required during handling, storage and transportation are built in the design of PAD. The cartridges are required to undergo different exhaustive design qualification tests to qualify design aspects. Total life of 6 years is assigned to cartridge after performance degradation study of the propellant which includes 2 years installed life. This paper explains about the development aspects of PAD, its use, function, testing and performance evaluation methodology in a suitable fabricated Velocity Test Rig (VTR). The maximum slug velocity is 121.14 m/s in hot condition and minimum slug velocity is 99.14 m/s in cold condition. The main objective of this paper is to device a novel method to measure actual slug velocity of the aircraft gun inside a cartridge using VTR and Doppler RADAR.

Digital Twin Driven Green Performance Evaluation Methodology of Intelligent Manufacturing: Hybrid Model Based on Fuzzy Rough-Sets AHP, Multistage Weight Synthesis, and PROMETHEE II

The design, planning, and implementation of intelligent manufacturing are mainly carried out from the perspectives of meeting the needs of mass customization, improving manufacturing capacity, and innovating business pattern currently. Environmental and social factors should be systematically integrated into the life cycle of intelligent manufacturing. In view of this, a green performance evaluation methodology of intelligent manufacturing driven by digital twin is proposed in this paper. Digital twin framework, which constructs the bidirectional mapping and real-time data interaction between physical entity and digital model, provides the green performance evaluation with a total factor virtual image of the whole life cycle to meet the monitoring and simulation requirements of the evaluation information source and demand. Driven by the digital twin framework, a novel hybrid MCDM model based on fuzzy rough-sets AHP, multistage weight synthesis, and PROMETHEE II is proposed as the methodology for the green performance evaluation of intelligent manufacturing. The model is tested and validated on a study of the green performance evaluation of remote operation and maintenance service project evaluation for an air conditioning enterprise. Testing demonstrates that the proposed hybrid model driven by digital twin can enable a stable and reasonable evaluation result. A sensitivity analysis was carried out by means of 27 scenarios, the results of which showed a high degree of stability.

Damage assessment of reinforced concrete moment resisting frames using performance-based seismic evaluation procedure

The performance-based seismic design has two primary concerns: (a) appropriate quantification of the uncertainties associated with the performance evaluation process, and (b) satisfactory characterization of the associated structural damage for direct incorporation into the design or performance evaluation methodology. This study deals with seismic damage assessment of moment resisting frames. Some damage indicators are introduced by using the nonlinear responses obtained at the output of nonlinear static procedures. Among these, some are possible extensions of previously defined static indicators and some are newly defined ones. The damage variables used in formulation are inelastic displacement, spectral displacement, spectral acceleration, and base shear. The introduced damage indicators address the plastic ductility, loss of strength, stiffness degradation, and dissipated energy of a structure. The variations in damage values for various drift criteria were used to associate with different performance levels.

Performance-based Seismic Assessment of Reinforced Concrete Moment Resisting Frame

The performance-based seismic design has two primary concerns: (a) appropriate quantification of the uncertainties associated with the performance evaluation process, and (b) satisfactory characterization of the associated structural damage for direct incorporation into the design or performance evaluation methodology. This study attempts to address these primary concerns by evaluating the performance of reinforced concrete frame using nonlinear static procedures. For this, fifteen-moment resisting frames designed following the guidelines of Indian seismic codes were subjected to different lateral load patterns. The seismic performance is investigated in terms of fundamental periods, roof displacements, interstory drift ratio, base shear, and modification factor and was compared with various performance limits. The obtained results showed disagreement with Indian seismic code provisions, especially, towards the fundamental time period, upper and lower bound values of base shear drift ratio and modification factor.

Evaluation of haptic devices and end-users: Novel performance metrics in tele-robotic microsurgery.

Here, we present performance evaluation methodology that distinguishes the performance of a haptic device from end-user skill level in a tele-robotic system. A pick-&-place experiment was designed and eight participants micromanipulated cotton strips, similar to maneuvers performed during microsurgery. Using three nonredundant haptic devices: neuroArmPLUS HD , a custom developed master manipulator, and two commercially available products, sigma.7 and HD2 , several features including the speed, effort, consistency, hand/gimbal agility, and force characteristics were measured and recorded for each participant and device. The participants showed variable skill level. For consistency, hand/gimbal agility and force characteristics, they performed significantly better when using neuroArmPLUS HD prototype. Based on the experimental data, performance metrics for both the device and the end-users were established. The integrated performance metrics allows independent evaluation of both the user and haptic device, thereby quantifying human-machine interactions.

Performance Evaluation Methodology for Long-Term Single-Object Tracking.

A long-term visual object tracking performance evaluation methodology and a benchmark are proposed. Performance measures are designed by following a long-term tracking definition to maximize the analysis probing strength. The new measures outperform existing ones in interpretation potential and in better distinguishing between different tracking behaviors. We show that these measures generalize the short-term performance measures, thus linking the two tracking problems. Furthermore, the new measures are highly robust to temporal annotation sparsity and allow annotation of sequences hundreds of times longer than in the current datasets without increasing manual annotation labor. A new challenging dataset of carefully selected sequences with many target disappearances is proposed. A new tracking taxonomy is proposed to position trackers on the short-term/long-term spectrum. The benchmark contains an extensive evaluation of the largest number of long-term trackers and comparison to state-of-the-art short-term trackers. We analyze the influence of tracking architecture implementations to long-term performance and explore various redetection strategies as well as the influence of visual model update strategies to long-term tracking drift. The methodology is integrated in the VOT toolkit to automate experimental analysis and benchmarking and to facilitate the future development of long-term trackers.

Gestão editorial de periódico científico de acesso aberto em consolidação: análise da Revista Latino-Americana em Avaliação do Ciclo de Vida (LALCA)

Periódicos científicos em consolidação, em geral, não contam fontes renomadas para geração e acompanhamento de indicadores de desempenho, uma vez que não são indexados por bases de dados tradicionais. Frente a essa problemática, este estudo visou a análise e a experimentação de indicadores, ferramentas e metodologias de avaliação de desempenho para Revista Latino-Americana em Avaliação do Ciclo de Vida, a qual se encontra em estágio de consolidação. A partir da relação de autores dos artigos já publicados pela LALCA, foram coletados dados complementares, tais como origem geográfica, titulação, afiliação, entre outros, além de dados relacionados às estatísticas de acesso e uso da Revista e de seus artigos, utilizando-se como fontes de dados as plataformas Open Journal Systems, Lattes, e Open Researcher and Contributor ID e outros perfis e/ou páginas pessoais ou profissionais, e o Google Analytics. O tratamento e a análise dos dados geraram indicativos que podem auxiliar o processo de gerenciamento do periódico e a sua consequente qualificação e consolidação, além de instigar a ampliação e o reforço de registro e curadoria de metadados úteis a gestores de periódicos em processo de consolidação

Indicators proposed by the Brazilian Ministry of Health for monitoring and evaluating oral health actions in the National Health System: documentary research, 2000-2017.

to present the indicators for monitoring and evaluation of oral health actions in the Brazilian National Health System (SUS), proposed in the period 2000-2017. documental research conducted on the Ministry of Health website regarding government guidelines on oral health monitoring and evaluation systems; the indicators were classified according to the following categories: access to care; resolutive capacity and continuity; and availability of oral health services. oral health indicators were identified in the following guidelines: 'Health Services Performance Evaluation Methodology Project', 'SUS Qualification Evaluation Program', 'National Program for Improving Primary Care Access and Quality', and 'SUS Performance Index'; most of them refer to access to services and resolutive capacity and continuity of care. oral health indicators in the four government guidelines identified provide important input for health management, but new indicators are needed for effective monitoring and evaluation of oral health actions.