Incomplete Pairwise Comparison Matrices Research Articles

How to choose a completion method for pairwise comparison matrices with missing entries: An axiomatic result

Since there exist several completion methods to estimate the missing entries of pairwise comparison matrices, practitioners face a difficult task in choosing the best technique. Our paper contributes to this issue: we consider a special set of incomplete pairwise comparison matrices that can be represented by a weakly connected directed acyclic graph, and study whether the derived weights are consistent with the partial order implied by the underlying graph. According to previous results from the literature, two popular procedures, the incomplete eigenvector and the incomplete logarithmic least squares methods fail to satisfy the required property. Here, the recently introduced lexicographically optimal completion combined with any of these weighting methods is shown to avoid ordinal violation in the above setting. Our finding provides a powerful argument for using the lexicographically optimal completion to determine the missing elements in an incomplete pairwise comparison matrix.

A new SMAA-based methodology for incomplete pairwise comparison matrices: Evaluating production errors in the automotive sector

Analysing and mitigating errors in production processes is a primary objective of companies in the automotive sector. Unfortunately, due to inaccurate or partially missing information, comparing errors is often very difficult, resulting from the experts’ provision of incomplete pairwise comparison matrices. In the literature, several techniques have been developed to complete such matrices. These techniques merely estimate what the decision makers or experts would have entered according to known entries. In this article, we propose a new methodology based on the stochastic multi-objective acceptability analysis; we apply it to vary the missing entries of the pairwise comparison matrix, thus providing the probability that an alternative/criterion will attain a given rank. This approach gives a complete view of the possible outcomes because it represents all possible decision maker mindsets. We present a case study carried out in a multinational automotive industry where we apply our methodology for evaluating errors in the production process.

On the coincidence of optimal completions for small pairwise comparison matrices with missing entries

AbstractIncomplete pairwise comparison matrices contain some missing judgements. A natural approach to estimate these values is provided by minimising a reasonable measure of inconsistency after unknown entries are replaced by variables. Two widely used inconsistency indices for this purpose are Saaty’s inconsistency index and the geometric inconsistency index, which are closely related to the eigenvector and the logarithmic least squares priority deriving methods, respectively. The two measures are proven to imply the same optimal filling for incomplete pairwise comparison matrices up to order four but not necessarily for order at least five.

The uncertainty related to the inexactitude of prioritization based on consistent pairwise comparisons.

When the in/consistency in Pairwise Comparisons (PCs) is taken into consideration as the subarea of the Multi Attribute Decision Making (MADM) scientific field, it has many repercussions in various types of research areas including different modelling scenarios e.g. reduction of inconsistency during PCs, deriving appropriate consistency thresholds for inconsistent Pairwise Comparison Matrices (PCMs), completing of incomplete PCMs, aggregating of individual PCMs in relation to Group Decision Making (GDM) aspects, and PCMs in/consistency relation to credibility of Priority Vectors (PV) derived from PCMs with the application of various Priorities Deriving Methods (PDMs). The examination objective in the latter area of research is the uncertainty related to the inexactitude of prioritization based on derived PVs. However, only few research studies examine this problem from the perspective of PCM applicability for credible designation of decision maker's (DM) priorities in the way that leads to minimization of the prioritization uncertainty related to possible, and sometimes very probable, ranking fluctuations. This problem constitutes the primary area of interest for this research paper as no research study was thus far identified that examines this problem from the perspective of consistent PCMs. Hence, a research gap was identified. Thus, the objective of this research paper is to fill in this scientific gap. The research findings have serious repercussions in relation to prioritization quality with the application of PCs methodology, mostly in relation to the interpretation and reliability evaluation of prioritization results. Firstly, the research study outcome changes the perspective of the rank reversal phenomenon, which shed new light on many research studies that have been presented in the subject's literature for many decades. Secondly, the research study results throw new light on the discussion concerning the fuzziness of AHP's results. Last but not least, the effect of the research opens the unique opportunity to evaluate the prioritization outcome obtained within the process of consistent PCs from the well-known perspective of statistical hypothesis testing i.e. the probability designation of the chance that accepted ranking results which were considered as correct due to low probability of change may be incorrect, hence they should be rejected, and the probability designation of the chance that rejected ranking results which were considered as incorrect due to high probability of change may be correct and should be accepted. The paramount finding of the research is the fact that consistent PCMs provide PVs, which elements cannot be considered as established, but only approximated within certain confidence intervals estimated with a certain level of probability. As problems related to heuristics can be analyzed only via a computer simulation process, because they cannot be mathematically determined, the problem examined in this research paper is examined via Monte Carlo simulations, appropriately coded and executed with the application of Wolfram's Mathematica Software. It is believed that this research findings should be very important and useful for all decision makers and researchers during their problems' examinations that relate to prioritization processes with the application of PCs methodology.

Enhancing Consistency of Pairwise Comparisons on the Base of Linear Algebraic Equations

A problem of improving consistency of pairwise comparisons matrices in application to ranking given alternatives is considered in the paper. But it can be shown that consistency is not the only issue as to the quality of pairwise comparisons. Given an arbitrary positive square matrix, we can obtain an ideally consistent pairwise comparison matrix with the same Perronian vector. Therefore, the quality of experts’ judgements is an issue of great importance as well.Technically, an approach to improving consistency of pairwise comparisons on the basis of solving a linear algebraic equations system is suggested. The system contains two groups of equations. One of them represents experts’ judgments, and the other is related to demands of cardinal consistency. Such a system can be over- or maybe underdetermined, and it typically can be inconsistent. Then a pseudo-solution can be obtained by means of pseudo-inverse Moore-Penrose matrix.For improving the quality of pairwise comparisons, it appears urgent to take into account reliabilities of certain judgements by giving them appropriate weight coefficients.Some numerical examples are provided in the paper. The first is a simple basic example without any serious inconsistencies. The second illustrates as to treat incomplete pairwise comparison matrices. And the latest illustrates possible expert’s manipulation, when an expert wants to secure the winning of a certain alternative whereas they don’t want to postulate the advantage of this alternative implicitly, and this results in the order violation. It is illustrated how introducing weight coefficients of equations can help counteract such manipulations.

Incomplete pairwise comparison matrices: Ranking top women tennis players

The method of pairwise comparisons is frequently applied for ranking purposes. This article aims to rank top women tennis players based on their win/lose ratios. Incomplete pairwise comparison matrices (PCMs) were constructed from data obtained from the WTA (Women’s Tennis Association) homepage. The database contains head-to-head results from the period between 1973 and 2022 for 28 players who had the position No. 1 in the official rankings of WTA. The weight vector was calculated from the incomplete PCM with the logarithmic least squares method and the eigenvector method. The results are not surprising: Serena Williams, Steffi Graf, and Martina Navratilova stand in the first three positions, and Martina Hingis, Kim Clijsters, and Justine Henin follow them. We also tested the frequently used probability-based Bradley-Terry method and found high rank-correlation values. Using graph representations, the results gave us a deeper insight into the properties of incomplete PCMs. Special attention was given to the nontransitive triads. A data modification was necessary to remove ties in order to apply the commonly used tests. The results indicate that ordinally nontransitive triads are not significant in the data we analysed.

Incomplete pairwise comparison matrices based on graphs with average degree approximately 3

A crucial, both from theoretical and practical points of view, problem in preference modelling is the number of questions to ask from the decision maker. We focus on incomplete pairwise comparison matrices based on graphs whose average degree is approximately 3 (or a bit more), i.e., each item is compared to three others in average. In the range of matrix sizes we considered, n=5,6,7,8,9,10, this requires from 1.4n to 1.8n edges, resulting in completion ratios between 33% (n=10) and 80% (n=5). We analyze several types of union of two spanning trees (three of them building on additional ordinal information on the ranking), 2-edge-connected random graphs and 3-(quasi-)regular graphs with minimal diameter (the length of the maximal shortest path between any two vertices). The weight vectors are calculated from the natural extensions, to the incomplete case, of the two most popular weighting methods, the eigenvector method and the logarithmic least squares. These weight vectors are compared to the ones calculated from the complete matrix, and their distances (Euclidean, Chebyshev and Manhattan), rank correlations (Kendall and Spearman) and similarity (Garuti, cosine and dice indices) are computed in order to have cardinal, ordinal and proximity views during the comparisons. Surprisingly enough, only the union of two star graphs centered at the best and the second best items perform well among the graphs using additional ordinal information on the ranking. The union of two edge-disjoint spanning trees is almost always the best among the analyzed graphs.

Inconsistency thresholds for incomplete pairwise comparison matrices

Pairwise comparison matrices are increasingly used in settings where some pairs are missing. However, there exist few inconsistency indices for similar incomplete data sets and no reasonable measure has an associated threshold. This paper generalises the famous rule of thumb for the acceptable level of inconsistency, proposed by Saaty, to incomplete pairwise comparison matrices. The extension is based on choosing the missing elements such that the maximal eigenvalue of the incomplete matrix is minimised. Consequently, the well-established values of the random index cannot be adopted: the inconsistency of random matrices is found to be the function of matrix size and the number of missing elements, with a nearly linear dependence in the case of the latter variable. Our results can be directly built into decision-making software and used by practitioners as a statistical criterion for accepting or rejecting an incomplete pairwise comparison matrix.

Filling in pattern designs for incomplete pairwise comparison matrices: (Quasi-)regular graphs with minimal diameter

Pairwise comparisons have become popular in the theory and practice of preference modelling and quantification. In case of incomplete data, the arrangements of known comparisons are crucial for the quality of results. We focus on decision problems where the set of pairwise comparisons can be chosen and it is designed completely before the decision making process, without any further prior information. The objective of this paper is to provide recommendations for filling patterns of incomplete pairwise comparison matrices based on their graph representation. The proposed graphs are regular and quasi-regular ones with minimal diameter (longest shortest path). Regularity means that each item is compared to others for the same number of times, resulting in a kind of symmetry. A graph on an odd number of vertices is called quasi-regular, if the degree of every vertex is the same odd number, except for one vertex whose degree is larger by one. We draw attention to the diameter, which is missing from the relevant literature, in order to remain the closest to direct comparisons. If the diameter of the graph of comparisons is as low as possible (among the graphs of the same number of edges), we can decrease the cumulated errors that are caused by the intermediate comparisons of a long path between two items. Contributions of this paper include a list containing (quasi-)regular graphs with diameter 2 and 3 up until 24 vertices. Extensive numerical tests show that the recommended graphs indeed lead to better weight vectors compared to various other graphs with the same number of edges. It is also revealed by examples that neither regularity nor small diameter is sufficient on its own, both properties are needed. Both theorists and practitioners can utilize the results, given in several formats in the appendix: plotted graph, adjacency matrix, list of edges, ‘Graph6’ code.

Constrained Eigenvalue Minimization of Incomplete Pairwise Comparison Matrices by Nelder-Mead Algorithm

Pairwise comparison matrices play a prominent role in multiple-criteria decision-making, particularly in the analytic hierarchy process (AHP). Another form of preference modeling, called an incomplete pairwise comparison matrix, is considered when one or more elements are missing. In this paper, an algorithm is proposed for the optimal completion of an incomplete matrix. Our intention is to numerically minimize a maximum eigenvalue function, which is difficult to write explicitly in terms of variables, subject to interval constraints. Numerical simulations are carried out in order to examine the performance of the algorithm. The results of our simulations show that the proposed algorithm has the ability to solve the minimization of the constrained eigenvalue problem. We provided illustrative examples to show the simplex procedures obtained by the proposed algorithm, and how well it fills in the given incomplete matrices.

Incomplete analytic hierarchy process with minimum weighted ordinal violations

ABSTRACT Incomplete pairwise comparison matrices offer a natural way of expressing preferences in decision-making processes. Although ordinal information is crucial, there is a bias in the literature: cardinal models dominate. Ordinal models usually yield nonunique solutions; therefore, an approach blending ordinal and cardinal information is needed. In this work, we consider two cascading problems: first, we compute ordinal preferences, maximizing an index that combines ordinal and cardinal information; then, we obtain a cardinal ranking by enforcing ordinal constraints. Notably, we provide a sufficient condition (that is likely to be satisfied in practical cases) for the first problem to admit a unique solution and we develop a provably polynomial-time algorithm to compute it. The effectiveness of the proposed method is analyzed and compared with respect to other approaches and criteria at the state of the art.

Incomplete Pairwise Comparison Matrices and Weighting Methods*

A special class of preferences, given by a directed acyclic graph, is considered. They are represented by incomplete pairwise comparison matrices as only partial information is available: for some pairs no comparison is given in the graph. A weighting method satisfies the linear order preservation property if it always results in a ranking such that an alternative directly preferred to another does not have a lower rank. We study whether two procedures, the Eigenvector Method and the Logarithmic Least Squares Method meet this axiom. Both weighting methods break linear order preservation, moreover, the ranking according to the Eigenvector Method depends on the incomplete pairwise comparison representation chosen.

Bridging the gap between missing and inconsistent values in eliciting preference from pairwise comparison matrices

Missing values and inconsistency of pairwise comparison matrices (PCM) in AHP are tackled with different methods in extant studies. In this article, we prove that connecting path method (CPM), which is a well-known approach of estimating missing judgments, also guarantees minimal geometric consistency index, and only involves elementary connecting paths by unraveling its connection with logarithmic least square method. The formal proofs suggest that CPM alone is sufficient to fix both missing judgments and inconsistency of PCM in AHP. Based on this well grounded merit, a procedure is proposed to improve consistency for incomplete PCM. The validity of the proposed procedure is illustrated by classical examples, and its advantages over other consistency improving approaches are also discussed in detail.

An application of incomplete pairwise comparison matrices for ranking top tennis players

Pairwise comparison is an important tool in multi-attribute decision making. Pairwise comparison matrices (PCM) have been applied for ranking criteria and for scoring alternatives according to a given criterion. Our paper presents a special application of incomplete PCMs: ranking of professional tennis players based on their results against each other. The selected 25 players have been on the top of the ATP rankings for a shorter or longer period in the last 40 years. Some of them have never met on the court. One of the aims of the paper is to provide ranking of the selected players, however, the analysis of incomplete pairwise comparison matrices is also in the focus. The eigenvector method and the logarithmic least squares method were used to calculate weights from incomplete PCMs. In our results the top three players of four decades were Nadal, Federer and Sampras. Some questions have been raised on the properties of incomplete PCMs and remains open for further investigation.

Minimization of the Perron eigenvalue of incomplete pairwise comparison matrices by Newton iteration

Abstract Pairwise comparison matrices are of key importance in multi-attribute decision analysis. A matrix is incomplete if some of the elements are missing. The eigenvector method, to derive the weights of criteria, can be generalized for the incomplete case by using the least inconsistent completion of the matrix. If inconsistency is indexed by CR, defined by Saaty, it leads to the minimization of the Perron eigenvalue. This problem can be transformed to a convex optimization problem. The paper presents a method based on the Newton iteration, univariate and multivariate. Numerical examples are also given.

First-Level Transitivity Rule Method for Filling in Incomplete Pair-Wise Comparison Matrices in the Analytic Hierarchy Process

The paper discusses the problem of performing the prioritization of decision elements within the multicriteria optimization method, analytic hierarchy process (AHP), with incomplete information. An approach is proposed on how to fill in the gap in the pair-wise comparison matrix generated within an AHP standard procedure; that is, to reproduce one missing judgment of the decision maker while assuring the reproduced judgment belongs to the same ratio scale used while other judgments are elicited. The first-level transitivity rule (FLTR) approach is proposed based on screening matrix entries in the neighborhood of a missing one. Scaling (where necessary) and geometric averaging of screened entries allows filling o f the gap in the matrix and later prioritization of involved decision elements by the eigenvector, or any other known method. Illustrative examples are provided to compare the proposed method with the other two known methods also aimed to fill-in gaps in AHP matrices. The results indicate some similarities in attaining consistency. However, unlike other methods, the FLTR assures coherency of the generating process in a sense that all numeric values in a matrix (original entries, plus one generated) come from the same ratio scale and have correct element-wise semantic equivalents.

Ranking by pairwise comparisons for Swiss-system tournaments

Pairwise comparison matrices are widely used in Multicriteria Decision Making. This article applies incomplete pairwise comparison matrices in the area of sport tournaments, namely proposing alternative rankings for the 2010 Chess Olympiad Open tournament. It is shown that results are robust regarding scaling technique. In order to compare different rankings, a distance function is introduced with the aim of taking into account the subjective nature of human perception. Analysis of the weight vectors implies that methods based on pairwise comparisons have common roots. Visualization of the results is provided by Multidimensional Scaling on the basis of the defined distance. The proposed rankings give in some cases intuitively better outcome than currently used lexicographical orders.

A parametric GP model dealing with incomplete information for group decision-making

Abstract We consider a group decision-making problem where preferences given by the experts are articulated into the form of pairwise comparison matrices. In many cases, experts are not able to efficiently provide their preferences on some aspects of the problem because of a large number of alternatives, limited expertise related to some problem domain, unavailable data, etc., resulting in incomplete pairwise comparison matrices. Our goal is to develop a computational method to retrieve a group priority vector of the considered alternatives dealing with incomplete information. For that purpose, we have established an optimization problem in which a similarity function and a parametric compromise function are defined. Associated to this problem, a logarithmic goal programming formulation is considered to provide an effective procedure to compute the solution. Moreover, the parameters involved in the method have a clear meaning in the context of group problems.

A Monte Carlo investigation of incomplete pairwise comparison matrices in AHP

The Analytic Hierarchy Process (AHP) is a decision analysis technique used to evaluate complex multiattribute alternatives among one or more decision makers. It imposes a hierarchical structure on any complex multicriterion problem. However, a major drawback of the AHP is that a large number of pairwise comparisons is needed to calibrate the hierarchy. When there are a few levels and sublevels, the AHP can be applied in a straightforward manner to derive the weights (relative preference for each alternative). As the size of the hierarchy increases, the number of pairwise comparisons increases rapidly. It is well established in the marketing and consumer behavior literature that in a very long interview, even under the best circumstances, the respondent is likely to suffer from information overload. Recognition of this problem was the motivation which led to the investigation of a modification of AHP which required less data collection, i.e., a reduction in the threat of information overload. The first question to be answered is the effect on AHP weights due to different patterns of missing data likely to result from reallife data collection. In this study, a Monte Carlo simulation was conducted, which uses the Incomplete Pairwise Comparisons (IPC) algorithm [14], to investigate the effect of reduced sets of pairwise comparisons in the AHP. Data for the study were generated with known structure and comparisons made between complete and incomplete matrices. The results of the simulation suggest that incomplete sets of pairwise comparison matrices can capture the attribute level weights without significant loss of accuracy and independent of decision model (form and amount of error) considered.