Relative Importance Of Evaluation Criteria Research Articles

E-Learning Platform Assessment and Selection Using Two-Stage Multi-Criteria Decision-Making Approach with Grey Theory: A Case Study in Vietnam

Education has changed dramatically due to the severe global pandemic COVID-19, with the phenomenal growth of e-learning, whereby teaching is undertaken remotely and on digital platforms. E-learning is revolutionizing education systems, as it remains the only option during the ongoing crisis and has tremendous potential to fulfill instructional plans and safeguard students’ learning rights. The selection of e-learning platforms is a multi-criteria decision-making (MCDM) problem. Expert analyses over numerous criteria and alternatives are usually linguistic terms, which can be represented through grey numbers. This article proposes an integrated approach of grey analytic hierarchy process (G-AHP) and grey technique for order preference by similarity to ideal solution (G-TOPSIS) to evaluate the best e-learning website for network teaching. This introduced approach handles the linguistic evaluation of experts based on grey systems theory, estimates the relative importance of evaluation criteria with the G-AHP method, and acquires e-learning websites’ ranking utilizing G-TOPSIS. The applicability and superiority of the presented method are illustrated through a practical e-learning website selection case in Vietnam. From G-AHP analysis, educational level, price, right and understandable content, complete content, and up-to-date were found as the most impactful criteria. From G-TOPSIS, Edumall is the best platform. Comparisons are conducted with other MCDM methods; the priority orders of the best websites are similar, indicating the robust proposed methodology. The proposed integrated model in this study supports the stakeholders in selecting the most effective e-learning environments and could be a reference for further development of e-learning teaching-learning systems.

Expertise-based consensus building for MCGDM with hesitant fuzzy linguistic information

The integration of a consensus reaching process (CRP) becomes paramount to make highly accepted group decisions in complex real-life multi-criteria group decision making (MCGDM) problems. Notwithstanding, existing CRPs for MCGDM do neither exhaustively analyse the diversity in decision makers’ expertise levels, nor they consider that (because of such diversity) individuals might exhibit distinct perceptions on the relative importance of evaluation criteria. In this study, we present a novel expertise-based consensus building model for MCGDM under a hesitant fuzzy linguistic setting. Firstly, an expertise identification approach is devised to objectively determine the expertise degree of each decision maker based on multiple features. The proposed approach allows to dynamically assigning importance weights to the decision makers’ opinions based on their expertise, as well as intelligently combining their individually elicited subjective and objective criteria weights into meaningful expertise-dependent combinative weights. Then, a CRP for MCGDM problems is introduced based on an improved consensus measurement process and an expertise-based feedback mechanism that provides a highly tailored, personalised means of direction rules to guide decision makers during the consensus building process. A numerical example is provided to illustrate the application of the CRP, and a detailed comparison analysis is presented to verify the validity and accuracy of this study’s proposal.

Model for Selection of the Best Location Based on Fuzzy AHP and Hurwitz Methods

The problem of evaluation and selection of parking lots is a part of significant issues of public transport management in cities. As population expands as well as urban areas, solving the mentioned issues affects employees, security and safety of citizens, and quality of life in long-time period. The aim of this paper is to propose a multicriteria decision model which includes both quantitative and qualitative criteria, which may be of either benefit or cost type, to evaluate locations. The criteria values and the importance of criteria are either precise or linguistic expressions defined by trapezoidal fuzzy numbers. The human judgments of the relative importance of evaluation criteria and uncertain criteria values are often vague and cannot be expressed by exact precise values. The ranking of locations with respect to all criteria and their weights is performed for various degrees of pessimistic-optimistic index. The proposed model is tested through an illustrative example with real life data, where it shows the practical implications in public communal enterprises.

A HYBRID MULTICRITERIA APPROACH FOR PERFORMANCE EVALUATION: THE CASE OF A HOLDING COMPANY

In today’s competitive business situation, performance evaluation of firms is an extremely important concern of all the people who are typically stakeholders of the business game. In case of holding companies, this is a more important issue since the parent firm must permanently control the situation of its subsidiaries in their sectors to make appropriate investment decisions. This paper develops a multicriteria decision making (MCDM) approach for evaluating performance of firms considering financial and productivity criteria. We adopt a Fuzzy Analytic Hierarchy Process (FAHP) method to determine the relative importance of evaluation criteria, taking the vagueness and imprecision of human judgments into consideration. Then, we employ the Preference Ranking Organization METHod for Enrichment Evaluation (PROMETHEE) for ranking of firms. Afterward, this paper enjoys benefit of using Technique of Order Preference by Similarity to Ideal Solution (TOPSIS) to assess the validity of the obtained ranking results. Our approach was applied to a holding company listed in Tehran Stock Exchange (TSE) as a real case. The analysis of ranking results revealed advantages of combining these MCDM methods.

Selecting infrastructure maintenance projects with Robust Portfolio Modeling

Project portfolios for the annual maintenance of infrastructure assets may contain dozens of projects which are selected out of hundreds of candidate projects. In the selection of these projects, it is necessary to account for multiple evaluation criteria, project interdependencies, and uncertainties about project performance as well as financial and other relevant constraints. In this paper, we report how Robust Portfolio Modeling (RPM) has been used repeatedly at the Finnish Transport Agency (FTA) for bridge maintenance programming. At FTA, project selection decisions are guided by the RPM's Core Index values which are derived from portfolio-level computations and reflect incomplete information about the relative importance of evaluation criteria. To-date, this application has been rerun with fresh data for six consecutive years. By drawing on experiences from this application, we discuss preconditions for the successful use of RPM or other methods of Portfolio Decision Analysis in comparable settings. We also develop an approximative algorithm for computing non-dominated portfolios in large project selection problems.

Multi-criteria group decision making for evaluating the performance of e-waste recycling programs under uncertainty

This paper presents a multi-criteria group decision making approach for effectively evaluating the performance of e-waste recycling programs under uncertainty in an organization. Intuitionistic fuzzy numbers are used for adequately representing the subjective and imprecise assessments of the decision makers in evaluating the relative importance of evaluation criteria and the performance of individual e-waste recycling programs with respect to individual criteria in a given situation. An interactive fuzzy multi-criteria decision making algorithm is developed for facilitating consensus building in a group decision making environment to ensure that all the interest of individual decision makers have been appropriately considered in evaluating alternative e-waste recycling programs with respect to their corporate sustainability performance. The developed algorithm is then incorporated into a multi-criteria decision support system for making the overall performance evaluation process effectively and simple to use. Such a multi-criteria decision making system adequately provides organizations with a proactive mechanism for incorporating the concept of corporate sustainability into their regular planning decisions and business practices. An example is presented for demonstrating the applicability of the proposed approach in evaluating the performance of e-waste recycling programs in organizations.

PHP132 VALUE OF MULTI-CRITERIA DECISION ANALYSIS IN EARLY ASSESSMENT OF MEDICAL DIAGNOSTIC DEVICES

OBJECTIVES: Multicriteria decision analytic (MCDA) techniques are a powerful tool in evaluating health care interventions where multiple, often competing, factors need to be considered. The analytic hierarchy process (AHP) is one such technique. We have applied AHP to evaluate medical diagnostic technologies as to support product development and market access. The main objective of this study is to show feasibility of AHP in terms of applicability and outcome. We therefore have studied the expected performance of the Photoacoustic Mammoscope (PAM), a new imaging device for detection of breast cancer based on photoacoustic imaging. METHODS: Before starting the study we defined different diagnostic and screening scenarios for the PAM including criteria that are important in the acceptance of the device. We finally defined four strategies including mammography, ultrasound and MRI imaging. The Analytic Hierarchy Process (AHP) analysis was then used to identify the relative importance of evaluation criteria, and to identify the relative performance of four different breast cancer imaging techniques. RESULTS: According to the expert panel the most important criterion in the design of a diagnostic breast imaging device is sensitivity (overall weight was 54%). Factors that mainly determine the performance on sensitivity are the visualization of mass margins, mass shape and vascularization. Second most important criterion was safety. CONCLUSIONS: From an early assessment of photoacoustic imaging it may be concluded that the potential clinical performance justifies further development and implementation of PAM. However, further research on the use of AHP in medical product development is required. We present and discuss some ideas to further integrate clinical evidence into MCDA using Bayesian techniques as well as sensitivity analyses on the model inputs.

Comparing the relative importance of evaluation criteria in proprietary and open-source enterprise application software selection - a conjoint study of ERP and Office systems

Until recently, organizations willing to acquire application systems have had no choice but to adopt proprietary software. With the advent of open-source software (OSS), a new model for developing and distributing software has entered the stage. OSS has evolved from a generally horizontal infrastructure towards more highly visible applications in vertical domains, giving information systems (IS) managers more degrees of freedom in their selection of enterprise application software (EAS). Although a large body of research exists on the relative importance of evaluation criteria for proprietary EAS, the role of OSS in the EAS evaluation process has received little attention so far. To address this research gap, this study represents the first empirical investigation to compare the relative importance of evaluation criteria in proprietary and open-source EAS selection. Through an online survey, we evaluated the responses of IS managers of 358 organizations to a conjoint study spawning 8592 trade-off pair comparisons and 3580 purchase evaluations on proprietary and open-source enterprise resource planning (ERP) and Office software packages. The results show that the relative importance of evaluation criteria significantly varies between proprietary and open-source ERP systems. Implementation factors such as ease of implementation and support are much more crucial in the evaluation of open-source than of proprietary ERP systems, which is generally due to IS managers' risk mitigation behaviour. Interestingly, there are no major differences in the ranking of evaluation criteria between proprietary and open-source Office systems. We conclude our paper with a detailed discussion of our findings and their implications for researchers, companies, EAS vendors and open-source communities.

Comparison of first aggregation and last aggregation in fuzzy group TOPSIS

The Technique for Order Preference by Similarity to the Ideal Solution (TOPSIS), one of the major multi attribute decision making (MADM) techniques, ranks the alternatives according to their distances from the ideal and the negative ideal solution. In real evaluation and decision making problems, it is vital to involve several people and experts from different functional areas in decision making process. Also under many conditions, crisp data are inadequate to model real-life situations, since human judgments including preferences are often vague and cannot estimate his preference with an exact numerical value. Therefore aggregation of fuzzy concept, group decision making and TOPSIS methods that we denote “fuzzy group TOPSIS” is more practical than original TOPSIS. There are two approaches for aggregating values including relative importance of evaluation criteria with respect to the overall objective and rating of alternatives with respect to each criterion in fuzzy group TOPSIS: (1) First aggregation, (2) Last aggregation. In first aggregation approach weight of each criterion and rating of alternatives with respect to each criterion gained from decision makers are aggregated at first and TOPSIS method then apply to these aggregate values. In last aggregation approach weight of each criterion and rating of alternatives with respect to each criterion gained from decision makers are used in TOPSIS method directly. Distance of each alternative from the ideal and the negative ideal solution are calculated then aggregated for finding relative closeness of each alternative to the ideal solution. Two examples are presented to highlight the procedure of the proposed methods at the end of this paper. We want to test if variation in decision makers’ opinions is high, last aggregation method is more precise than first aggregation and vice versa when this variation is low, first aggregation method is as precise as last aggregation but faster than this method.

A fuzzy approach to evaluation and management of therapeutic procedure in diabetes mellitus treatment

In this paper a new fuzzy model (FMOTPD2) is developed and by this model the measures of beliefs are determined so that one of the groups of possible therapeutic procedures is optimal for each patient of type 2 diabetes on hospital treatment. The choice of therapeutic procedure on individual level, which is one of the demands of modern medicine, means that each therapeutic procedure is to be evaluated by multiple and different criteria. In this paper, evaluation criteria are classified into two groups: (1) common criteria by which medicines used by the type 2 diabetes patients are being evaluated and (2) specific criteria, by which the patients' 1h state of health with type 2 diabetes mellitus is being estimated. Generally, the relative importance and values of these criteria are different. It is assumed that (a) the relative importance of evaluation criteria is defined by a team of medical experts and described by linguistic expressions and (b) the values of evaluation criteria are determined by evidence data, anamnesis and a diagnostic process. They can be crisp or uncertain. The most often used linguistic expressions describing the relative importance of evaluation criteria are modeled by triangular fuzzy numbers. The rest of uncertainties, which exist in developed model are described by discrete fuzzy numbers. A new algorithm for determining a unified fuzzy portrait of treated therapeutic procedures for each patient is given. It enables calculation of the measures of beliefs that some therapeutic procedures are more optimal than the others. The developed model is illustrated by examples with real word data collected in a hospital.

FMS selection based on area method under fuzzy environment

A number of feasible alternative Flexible Manufacturing Systems (FMSs) configurations are available when a manufacturer seeks for it. However, determining the best configuration from the feasible ones is a critical issue because the improper selection of the FMSs may adversely affect profitability. To effectively select an FMS, several factors have to be considered. The Multicriteria Decision Making (MCDM) approach is often used to solve various decision making and/or selection problems. This approach often requires the decision makers to provide qualitative and/or quantitative assessments to determine the performance of each alternative with respect to each criterion and the relative importance of evaluation criteria with respect to the overall objective. In the past, the contexts of research covered the spectrum of managerial issues from focusing on cost management system to the application of advanced mathematical models to understand the FMS and its characteristics. The conventional methods of FMS selection are inadequate in dealing with the vague nature of linguistic assessment. To overcome this difficulty, a fuzzy MCDM approach based on area method is being proposed to systematically evaluate alternative FMSs by considering both subjective and objective criteria. A numerical example is used to illustrate the efficiency of the proposed approach.

A multi-methodological approach for shipping registry selection in maritime transportation industry

Selecting the most suitable registry alternative for shipping fleets has become a highly complex strategic activity for ship owners in the maritime transportation industry, and necessitates consideration of both internal and external environmental factors, which can be both qualitative and quantitative. The aim of this paper is to propose an originally structured multi-methodological approach based on the systematic application of the SWOT analysis, the AHP and the TOPSIS methods to support the critical decision process on shipping registry selection under multiple criteria. In the proposed approach, the SWOT analysis is utilized to determine the key assessment factors on the shipping registry decision and to structure the decision hierarchy. While the AHP is used to measure the relative importance of evaluation criteria in this decision hierarchy, the TOPSIS is applied to rank the shipping registry alternatives. As a case study, major open registries are analyzed to select the most suitable of them for the fleet that is operated by a worldwide ship owner. The results show that the proposed multi-methodological approach takes the advantages of the SWOT analysis, the AHP and the TOPSIS to select the most suitable shipping registry. The proposed approach also provides a relatively simple and very well suited decision-making tool for this type of strategic decision making problem.

Using Decision Modeling with Stakeholders to Reduce Human–Wildlife Conflict: a Raptor–Grouse Case Study

Abstract: The successful resolution of human‐wildlife conflicts requires the participation of local communities and other stakeholder groups in formulating management decisions. In the uplands of the United Kingdom, a controversial conservation issue concerns the relationship between the conservation of a legally protected raptor, the Hen Harrier (Circus cyaneus) and the management of a gamebird, the Red Grouse (Lagopus lagopus scoticus). We used multicriteria analysis to evaluate the perspectives of two groups of stakeholders, grouse managers and raptor conservationists, and the acceptability to them of different management solutions to this conflict. Both groups quantified the relative importance of evaluation criteria and used these as a basis for comparing different upland and Hen Harrier management options. In relation to upland management, grouse managers placed more importance on economic criteria than did raptor conservationists, who valued natural‐environment criteria more highly. Intensively managed grouse moors, involving the control of harrier numbers, were ranked most highly by grouse managers and managed nature reserves by raptor conservationists, but both groups also ranked legally managed grouse moors highly. When evaluating Hen Harrier management options, grouse managers considered time scale and cost the most important criteria, whereas raptor conservationists considered the effects on harrier populations to be most important. Harrier quota schemes were the management solution most favored by grouse managers, whereas raptor conservationists preferred allowing harriers to attain natural densities. Notably, however, one technique that has already been partly tested in the field—the use of diversionary feeding was scored highly by both groups and thus holds promise for some form of compromise. This exercise highlighted the value of these objective techniques for developing dialog and trust between stakeholder groups, and it highlighted the need to conduct further research to test the effectiveness of different management options. There was broad agreement that the workshop moved the prior positions of individual stakeholders and was a valuable tool in helping to resolve human‐wildlife conflicts.

Use of fuzzy set theory in the aggregation of expert judgments

It is sometimes impossible to make a correct decision in a certain engineering task without the help from professional expert judgments. Even though there are different expert opinions available, however, they should be appropriately aggregated to a useful form for making an acceptable engineering decision. This paper proposes a technique which utilizes the fuzzy set theory in the aggregation of expert judgments. In the technique, two main key concepts are employed: linguistic variables and fuzzy numbers. Linguistic variables first represent the relative importance of evaluation criteria under consideration and the degree of confidence on each expert perceived by the decision maker, and then are replaced by suitable triangular fuzzy numbers for arithmetic manipulation. As a benchmark problem, the pressure increment in the containment of Sequoyah nuclear power plant due to reactor vessel breach was estimated to verify and validate the proposed technique.

Validation of the venture evaluation model in Korea

This article examines the validity of the venture evaluation model in Korea by directly comparing the relative importance of evaluation criteria on the funding decision with the relative importance of factors influencing venture's empirical performance. Specifically, we first identify the relative importance of evaluation criteria based on whether a deal is accepted or rejected (current evaluation model). Next, we derive the relative importance of those criteria revealed in the empirical performance, based on the success/failure of ventures that overlap with the deals originally used as accepted cases (performance-based model). Finally, to better understand the differences between the two models, we measure the general perceptions of venture capitalists on the importance of those criteria without regard to specific deals (normative model). To identify potentially important criteria, we reviewed the existing related studies on evaluation criteria, and we interviewed 10 venture capitalists. As a result, we selected 31 evaluation items. The questionnaire consists of two parts and deals with the following four cases: In part A: 1. 1. case 1: general perceptions concerning the 31 criteria. In part B: 2. 2. case 2: one of the most successful ventures. 3. 3. case 3: one of the least successful ventures. 4. 4. case 4: one of the rejected ventures. Using a five-point scale, we asked the respondents to rate the 31 criteria based on their general perceptions without referring deals (case 1), and on the specific deals they had handled (cases 2, 3, and 4). To prevent any firm or respondent from affecting the results more than it ought to, we distributed the questionnaire to each firm in proportion to the investment amounts; we also asked the respondents to fill out only one questionnaire including four cases. Seventy-four venture capitalists (including 10 interviewed) responded to this questionnaire. They have worked as investment directors or managers in the top-rated 20 venture capital companies, selected according to investment amounts. On the basis of the ratings of 31 criteria for the previous four cases, we developed three investment models. First, the current evaluation model reveals which criteria are more important in deciding whether or not to fund. The model measures the relative importance of evaluation factors in discriminating between the accept (case 2 and case 3) and the reject (case 4) decision. Second, the performance-based model consists of the relative weights of evaluation factors influencing the two performance categories—success (case 2) and failure (case 3). This model shows the relative importance of evaluation criteria that are reflected in the venture's empirical performance. Finally, we identify the normative model, revealing the venture capitalists' general perceptions concerning the importance of individual criteria in light of their past field experiences (case 1). By comparing the three models, we find that the importance of these criteria revealed from venture capitalists' perceptions and empirical performances is fairly similar but not sufficiently reflected in the actual deal evaluation. In particular, the entrepreneur-related characteristics are not considered the most important in evaluating venture proposals according to the current evaluation model, but they do turn out to be the most important factors in evaluating empirical performance. This is in strong contrast to the findings of some existing studies, which contend that the entrepreneurrelated criteria ultimately determine the funding decision. Instead, the result confirms the previous assertion that the evaluation of managerial capability is the most challenging task in the venture selection process. In the current evaluation model, financing ability is considered more important than either superiority of product and technology or production capability, contrary to venture capitalists' perceptions and empirical performance results. Therefore, it seems necessary to raise the level of specialization of venture capital firms through the acquisition of managerial skills from developed countries and by enhancing the technical background of individual venture capitalists.

Die relatiewe belangrikheid van verskillende evalueringskriteria by die koop van 'n passasiersvoertuig: 'n Vergelykende studie van mans en vroue

A number of studies revealed that the role of women both as influencer and independent decision-maker is becoming increasingly important in the buying decision involving the purchase of a passenger vehicle. In today's highly competitive South-African motor vehicle market it is becoming vital that motor manufacturers pay greater attention to market needs - specially with regard to important market segments. This study analyses certain aspects of the market segment consisting of women. The null-hypothesis this study tested was that there are no differences between male and female consumers pertaining to the relative importance of evaluation criteria, when purchasing a passenger vehicle. By means of analysis of variance it was established that there are significant differences (P0,01) between men and women with regard to 15 of the 65 evaluation criteria. It was also found that there are 50 insignificant differences between the two groups. The hypothesis, as formulated is thus rejected, with regard to 15 of the evaluation criteria, in favour of the alternate hypothesis.

Rassegroepe se evaluering van kriteria by motovoertuigaankope

The objective of this study was to determine whether differences and/or similarities exist in the relative importance of evaluation criteria between the various race groups in South Africa when purchasing a motor vehicle. The sample consisted of 577 whites, 114 coloureds, 102 Asians and 109 blacks mainly from metropolitan areas. Variance analysis was applied to establish that there were 52 significant differences out of a total of 65 evaluation criteria with respect to the various race groups. Results showed that the biggest significant differences occurred between whites and blacks (66% of the evaluation criteria) and between whites and Asians (43% of the evaluation criteria).

Die relatiewe belangrikheid van keuringskriteria soos waargeneem vanuit die oogpunt van verbruikersbankbestuurders

The relative importance of evaluation criteria as observed by bank managers in consumer banking This study has the following objectives: (i) To determine the relative importance of various evaluation criteria (as observed by bank managers) in considering the granting of credit to individuals in consumer banking; and (ii) to compare the relative importance of evaluation criteria (as observed by bank managers) with regard to the various race groups (coloureds, Indians, whites, and blacks). The sample consists of 510 managers from the consumer division of general and commercial banks. It was found that significant differences exist regarding the relative importance of the evaluation criteria (according to the perception of bank managers) with respect to the various race groups.