ELECTRE ME: A PROPOSAL OF AN OUTRANKING MODELING IN SITUATIONS WITH SEVERAL EVALUATORS

Highlights: This paper describes an original proposal for modeling Multicriteria problems taking into account more than one evaluator. It allows each evaluator to have its own set of criteria. It also avoids the incoherency of adopting compensatory techniques into non-compensatory algorithms. Goal: This paper describes an original proposal for modeling multicriteria situations where multiple evaluators take part of the evaluation process. This proposal allows each evaluator to have its own set of criteria, including their weights, and also avoids the usual inconsistency of adopting pre-processing compensatory methods for introducing it into non-compensatory algorithms. Design / Methodology / Approach: In order to better describe how ELECTRE ME works, a multicriteria-multiple evaluator situation is modeled by ELECTRE TRI ME (as we have called the ELECTRE TRI variation that incorporates the principles of multiple evaluators). Results: ELECTRE ME was able to avoid the inconsistency of adopting contradictory mechanisms of aggregating preferences while modeling multicriteria & multiple evaluators problems (first called here as MCDA-ME). Limitations: Although the proposal focuses in situations with multiple evaluators, there is no restriction for its application in situations where there is only one decision maker. Practical implications: Another important feature of ELECTRE ME is that it allows each evaluator to consider its own set of criteria and its own scale for evaluation. Originality / Value: ELECTRE ME avoids a contradictory approach to use compensatory algorithms (such as weighted mean) as an input in non-compensatory outranking methods. Despite the fact that non-compensatory principle is in the heart of the ELECTRE methods, it has not found a previous proposal with the attributes shown in this study: to incorporate outranking concepts in situations where more than one evaluator is present and, by extension, allow each evaluator to have its own set of criteria.


INTRODUCTION
While dealing with group decision or evaluation situations in which the opinions or perceptions of several or multiple evaluators appears, there are two mains streams and basic approaches: the consensus and the voting systems. The consensus is reached through dialogue and, as reported in Sobral and Costa (2012), the first approach can produce good solutions, once it usually incites discussions and induces a better comprehension about the problem. Unfortunately, the dialogue and interactive-discussion-based approaches are limited to a small number of evaluators.
On the other hand, are the voting systems, designed for situations where there is no way to adopt a consensus through a dialogue approach, such as when there is a great number of evaluators, which makes the use of consensus approaches based on dialogue unaffordable. In this second type of situation, there is an attempt to apply a voting system and consolidate the different evaluations in an overall number that should be an approximation of the consensus among the evaluations as a whole. There are several models that apply more refined techniques to reach such overall number, as a sample cited in: Nordström et al. (2009), Yu andLai (2011), Leyva López andAlvarez Carrillo (2015), Pereira and Costa (2015), Sant'Anna et al. (2016), Ding et al. (2017), Zeng et al. (2018) and Wu and Liao (2019). Despite this fact, in the voting systems, it is still usual to apply a weight sum algorithm for aggregating the preferences.
Following another stream, there are situations with unitary evaluations (evaluations made by only one evaluator or a group of evaluators whose opinions are expressed by a unique number or evaluation). For such situations, Roy (1968a) proposed the outranking approach, that is in the basis of ELECTRE (ELimination Et Choix Traidusaint la REalité) family of MCDA (Multicriteria Decision Aid) methods. If the seminal references that appear in Table 1 are analyzed, it could be concluded that ELECTRE methods were proposed for situations with unitary evaluations. As reported in Roy (1968a), such outranking procedure is so-called a non-compensatory method, once it focuses on eliminating the compensatory undesirable effects that appear in algorithms usually adopted for aggregating the Decision Maker (DM) evaluations, such as the weighted means. Compensatory effects also appear in other Multicriteria methods, such as Topsis (Hwang et al., 1993) and AHP (Saaty and Vargas, 2006). Despite the outranking appeal of ELECTRE methods, when dealing with opinions or evaluations that come from different evaluators, the ELECTRE-based models usually adopt a compensatory approach in order to get an overall number from aggregating individual preferences and introduce such number as an input to the non-compensatory outranking ELECTRE procedure, as it occurs in Galo (2015), among others. The main problem observed in this last approach is that the determination of this overall number is generally based in a compensatory aggregation method, which is not in agreement with the non-compensatory principles of ELECTRE method.
The main question that arises from this problem is: How to deal with the opinions from different evaluators when there is no way to find out consensus? This paper aims to describe an original and simple variation on the usual ELEC-TRE methods, to deal simultaneously with both multicriteria and multiple decision maker situations, and that also incorporates the non-compensatory and non-dominance principles of ELECTRE while dealing with multiple decision makers evaluations. Roy (1968a) proposed the ELECTRE method, that is based on non-compensatory outranking principles. Costa et al. (2007) established an analogy with a volleyball game in order to provide a more comprehensive explanation about the main differences between a compensatory approach (such as the weight sum) and a non-compensatory outranking approach. Thus, in order to well-establish the difference between a compensatory and a non-compensatory approach, it should be considered an analogy with a volleyball match, in which team A wins team B by 25 to 5 in the first set, but loses all the three following sets to team B by 25 to 20. In this situation, one could then imagine the following procedures to identify the winner of the match: a) A compensatory approach: Uses the sum of the points gained by the team in each set. In this case, A would be the winner by 85x80 points.

BACKGROUND: THE FUNDAMENTALS OF ELECTRE
b) A non-compensatory approach: Uses the number of game sets that each team wins. In this case, team B would be the winner by 3x1 sets. This procedure, which is actually adopted in volleyball matches, could be classified as an outranking approach.
The main outranking principle is very similar to the second procedure described above, if each game set is considered as a criterion.
Since the first appearance of ELECTRE in Roy (1968a) a large variety of MCDA methods have followed the non-compensatory outranking principles that are in the core of ELEC-TRE - Table 1 shows some of these variations. As it can be seen, ELECTRE TRI (Mousseau et al., 2000) and its variants ELECTRE TRI-C (Almeida-Dias et al., 2010) and TRI-nC (Almeida-Dias et al., 2012) are the latest members of the ELECTRE Family of MCDA methods.
It is interesting to notice that ELECTRE II, III, and IV are classified as ranking methods because they first and foremost build a rank, even though they are usually adopted to solve choice problems.

THE PROPOSAL: ELECTRE ME
The proposal reported in this paper was designated as ELECTRE ME (ELECTRE Multi Evaluators), in order to distinguish it from previous ELECTRE methods. The following assumptions were made in ELECTRE ME: • Alternatives • A= {a 1, a 2 , ... , a i , ..., a m } is a set composed by m alternatives.
• Fe j = {k 1ej, k 2ej , ..., k yej } is the subset composed by the y criteria adopted by the evaluator e j .
• Fe n = {k 1en, k 2en , ..., k zen } is the subset composed by the z criteria adopted by the evaluator e n .
• Weight of the criteria • We 1 = {w 1e1, w 2e1 , ..., w ve1 } is the subset composed by the weights of the v criteria under the per-spective of the evaluator e 1 . So that, in this vector, w 2e1 means the weight of criterion 2, under the opinion of evaluator e 1 .
• We 2 = {w 1e2, w 2e2 , ..., w xe2 } is the subset composed by the weights of the x criteria under the perspective of the evaluator e 2 .
• We j = {w 1ej, w 2ej , ..., w yej } is the subset composed by the weights of the y criteria under the perspective of the evaluator e j .
• We n = {w 1en, w 2en , ..., w zen } is the subset composed by the weights of the z criteria under the perspective of the evaluator e n .
• Performance of the alternatives .., g ve1 (a i )} is the subset composed by the performance of alternative a i under the perspective of the evaluator e 1 and the set of v criteria adopted by this evaluator. So that, in this vector, g 2e1 (a i ) means the performance of alternative a i under criterion 2, under the viewpoint of the evaluator e 1 .
• Ge 2 (a i ) = {g 1e2 (a i ) , g 2e2 (a i ), ..., g xe2 (a i )} is the subset composed by the performance of alternative a i under the perspective of the evaluator e 1 and the set of y criteria adopted by the evaluator e 2 .
• Ge j (a i ) = {g 1ej (a i ) , g 2ej (a i ), ..., g yej (a i )} is the subset composed by the performance of alternative a i under the perspective of the evaluator e 1 and the set of y criteria adopted by the evaluator e j .
• Ge n (a i ) = {g 1en (a i ) , g 2en (a i ), ..., g zen (a i )} is the subset composed by the performance of alternative a i under the perspective of the evaluator e 1 and the set of z criteria adopted by the evaluator e n .
The present proposal differs from the previous ELEC-TRE-based approaches in two ways: • It allows each evaluator to have its own set of criteria, and is its own scales for the evaluation of criteria weights and even alternative performances.
• It makes use of a non-compensatory approach to deal with the individual preferences of the evaluators.
To do this, it makes a unique and simple assumption: it assumes the whole criteria set (F) of overall criteria as the union of every subset from each evaluator. So that: where • Fe 1 is the subset of criteria adopted by evaluator e1 • Fe 2 is the subset of criteria that evaluator e2 considers relevant … … • Fe n is the subset of criteria that evaluator e3 takes into account.
where G(a) is the overall performance of a generic alternative a ε A.
After these assumptions, an ELECTRE ME-based model plays in the same way the previous ELECTRE models do.

THE ELECTRE TRI ME
Without losing generality, from this point, the focus is shifted to describing ELECTRE TRI ME: an outranking sorting method that adds the ME principles to ELECTRE TRI (Yu, 1992;Mousseau et al., 2000). The ELECTRE TRI ME was chosen as a background to show the ME adaptation of ELECTRE methods, since ELECTRE TRI-based methods are the most recent variations of ELECTRE non-compensatory method (as shown in Table 1). Therefore, the authors highlight that this decision does not result in any loss of generality and that the reasoning proposed in this study, by extension, could be applied to all the other ELECTRE methods. Next sections focus on presenting a short introduction to ELECTRE TRI, followed by an insertion about ELECTRE TRI ME and two numerical samples.

A briefing on the ELECTRE TRI
As reported in Mousseau et al. (2000) and shown in Figure 1, ELECTRE TRI sorts an alternative into a category from a set C of categories. These categories are delimited by a set of profiles or borders B, which are defined for each criterion, as it can be seen in Figure 2. The categories are ranked from the worst (C 1 ) to the best (C p+1 ). Observe that a generic profile b h is both the superior limit of C h , and the inferior limit of C h+1 .

570
The following two steps are played when running ELEC-TRE TRI for sorting the alternatives on the categories: • To build outranking relationship S, among the alternatives to be sorted and the profiles or boundaries of the categories.
• To exploit the relation S in order to assign each alternative to a category.
The statement aSb h means that "alternative a does not have performance worse than the profile b h ". In the validation of this statement aSb h , a credibility degree σ (a, b h ) is calculated, so that it expresses that the confidence with the statement "a is not worse than b h " is calculated. To define the outranking relation, a credibility cut-plane λ, is adopted. So that: The credibility degree σ(a, b h ) is calculated as it follows: In these equations: • c j (a, b h ) expresses the concordance degree with the statement "a is not worse than b h under the criterion g j " • d j (a, b h ) expresses the discordance degree with the statement "a is not worse than b h under the criterion g j " • q, p and v express, respectively, the indifference, preference, and veto threshoulds.
To assign an alternative to a category, the ELECTRE TRI method defines two sorting procedures: • The pessimistic (or more exigent) sorting procedure, described as follows: • Compare a successively with b j , for j = p; p -1, …, 1.
• Consider b h as the first profile limit so that aSb h , and classify a into class C h+1 (denoted by a C h+1 ).
• If there is no one profile b j such that aSb j , then a is classified into the lowest class: Class C 1 .
• The optimistic sorting procedure (or disjunctive) is described as follows: • Compare b j , j = 1; 2, …, p successively with a.
• Consider b j as the first profile such that b j > a, and classify a into class C h (denoted by a C h ).
• If there is no one profile b j such that b j Sa, then a is classified into the highest class: Class C p+1 .

Adding ME principles into ELECTRE TRI: the ELECTRE TRI ME
Thus, in ELECTRE TRI ME it is necessary to add two new assumptions to those previously shown: • C= {C 1, C 2 , ..., C h , ..., C p+1 } is the set of p categories that are ranked from the worst (C 1 ) to the best (C p+1 ).
• B kej = {b 1kej, b 2kej , ..., b hkej , ..., b pkej } is a set composed by p profiles or boundaries that delimit the categories for a criterion k, under the view point of an evaluator e j . As the categories are adjacent, b h is both the superior limit of C h , and the inferior limit of C h+1 .
In a more general way, taking into account that there should exist n evaluators: B = Be 1 U Be 2 U … U Be j U … U Be n-1 U Be n where B is the overall vector of profiles, and p is the number of profiles.
After these assumptions, ELECTRE TRI ME performs in the same way that ELECTRE TRI does.

Numerical example 1: a complete sample of applying ELECTRE TRI ME
Now take into account a situation where a meal X had its performance evaluated by three evaluators (E1, E2 and E3), in order that X could be assigned to one category of performance. Table 2 summarizes the data of this numerical sample, which is commented as follows: • The set A of alternatives is unitary, that is, it has only one alternative: X.
• The set of categories comprehends five possibilities: Very good, Good, Middle, Poor, and Very poor. So that: C = {VG; G; M; P; VP} • Each evaluator has its own criteria set and criteria weights, so that: Each evaluator has its own perception about the performance of X under its own criteria set.
• G e1 (x)={8.5; 6.8} Each evaluator should have its own perception about the boundaries of each class in C. So that: • B Appeal e1 ={2; 4; 6; 8} • B Taste e1 ={4; 6; 8; 9} • B Taste e2 ={2; 6; 8; 9} • B Proteins e2 ={4; 6; 8; 9} • B Vitamins e2 ={4; 6; 8; 9} • B Aroma e3 ={6; 7; 8; 9} An extensive search on Scopus data base, looking for previous works in MCDA fields was performed and it was not found in the literature a previous MCDA-based model with more than one evaluator in which each evaluator uses its own set of criteria and its own scale while evaluating the performance of alternatives. In the scope of ELECTRE TRIbased models, the fact that each evaluator uses its own set of profiles in order to define the limits of the categories in C is also unheard. Although in some decision situations it should be necessary to standardize these parameters, there are some situations in which it is interesting that the evaluators have independence while establishing such parameters, as when catching the costumers' perceptions regarding the quality of a service. Volume 15, Número 4, 2018, pp. 566-575 DOI: 10.14488/BJOPM.2018 As shown in figure 3, comparing the evaluations in G(x) vector with the profiles in B, it is possible to conclude that:

Brazilian Journal of Operations & Production Management
• E1 classifies the meal X into the category Very Good, while analyzing X's appeal, and classify X as Middle while evaluating it under the Taste criterion.
• E2 classifies X as Good under the criteria Vitamins and in the category Very Good under the Taste viewpoint.
• X is classified as Poor by evaluator E3 that has taken into account only one criterion: Aroma.  Figure 3. Performance of X under the criteria set and an ELECTRE TRI ME perspective.
From this point, we assume the thresholds p = q= 0, and no veto restriction. This assumption does not imply in any restriction or loss of generality to the description of ELECTRE TRI ME whatsoever.
The meal X is at least Very Good in two criteria (Appeal for evaluator E1 and Vitamins for E2), whose weights sum 12. Since 40 is the sum of all the weights in the vector W, the credibility degree in which X is at least Very Good is equal to 12/40; e.g. σ(X,VG) = 0.3.
Applying analog reasoning, it results in X being at least Good in five criteria (Appeal for E1; Taste, Protein, and Vitamins for E2; and, Aroma for E3), whose weights sum 32. Thus, the credibility degree in which X is at least Good is equal to 32/40; e.g. σ (X,G) = 32/40 = 0.8. It also results in σ (X,M) = σ (X,P) = σ (X,VP) = 40/40 = 1.0. Therefore, adopting a cutting plane sequal to 1.0, the meal X is classified into category Middle. However, if one wishes to adopt a less exigent cutting plane, such as σ=0.8, X would be classified as Good, since σ (X,G) = 0.8 ≥ σ.

Numerical example 2: comparing the results from ELECTRE TRI and ELECTRE TRI ME
Now, let us take into account another data set that refers to a situation where three evaluators (E1, E2 and E3) consider the same set of three criteria (C1, C2 and C3). Table 3 shows the performance of a generic alternative Y, and also the criteria importance, under the viewpoint of the evaluators. The data in Table 3 show that all the three evaluators (E1, E2 and E3) have assigned a weight equal to 1.00 to all the criteria, and they show that evaluators E1 and E2 agree that the performance of the alternative Y is equal to 2.50, in all the criteria. In order to simplify the explanation and without loss of generality, it was assumed that, for all criteria and all categories, p=q=0, and also the veto threshold was not taken into account. In this example, three categories are considered: C A , C B , and C C , where C A is better than C B that is better than C C . Table  4 shows the profiles that are the border of these categories. For simplicity and without loss of generality, in this example it was considered that the boundaries of the categories are the same for all the criteria.  Número 4, 2018, pp. 566-575 DOI: 10.14488/BJOPM.2018 Figure 4 shows an overview of the data: the performance of the alternatives in each criterion and the categories and their boundaries.

Performance of Y Evaluator E1
Evaluator E2 Evaluator E3 g 1e1 g 2e1 g 3e1 g 1e2 g 2e2 g 3e3 g 1e3 g 2e3 g 3e3 Category

Applying the traditional ELECTRE TRI
In situations where there is more than one evaluation for each criterion, it is usual to assume that, for each criterion, the mean from all evaluations could represent the evaluations of all evaluators, as a whole, so that, for example, the performance of the alternative Y should be represented by the following vector: G(Y) = {2.17; 2.17; 2.17} These values in vector G(Y) were calculated as the mean of the evaluations that come from of the three evaluators, for each criterion (C1, C2 and C3). This vector means that Y has the median of its performance equal to 2,17 in the three criteria, taking into account the perceptions of the three evaluators.
Thus, if one uses this mean as the evaluation of alternative Y, there is a credibility degree equal to 1.0 that Y outranks the floor of category C A ; e.g. σ(YSC A ) = 1.00. Therefore, ELECTRE TRI would classify Y into category C A without hesitation or doubt, by using the mean as the value to match the performance of the alternative.

Applying the ELECTRE TRI ME
On the other hand, while applying ELECTRE TRI ME, it is not necessary to aggregate the evaluations into a unique number that represents the evaluators' opinions as a whole. Table 5 shows the performance of the alternatives. Thus, the result from computing the credibility index is: σ(YSC A ) = 7/9 = 0.78 and σ(YSC B ) = 1.00. Therefore, by maintaining the same credibility cut-level adopted in the previous ELECTRE TRI modeling (λ=1.00), Y would be classified into category C B.

Comparing the results
As one can see, the results shown in section 4.2 differ from those in section 4.3. This is because in section 4.3 the use of the mean as a representation of the evaluations from the overall evaluators introduces a compensatory effect, which does not agree with the non-compensatory outranking principles that are in the basis of all the ELECTRE TRI methods. On the other hand, the results in section 4.4, generated by the ELECTRE TRI ME, preserve the outranking principle of ELECTRE methods, since it does not adopt a compensatory way to deal with the evaluations that come from different evaluators.

CONCLUSION
This paper has described the ELECTRE ME, an original variation of the ELECTRE methods. This method is able to incorporate the non-compensatory principles of ELECTRE while dealing with both multicriteria and multiple decision makers' evaluations.
The two main contributions of this approach are: • It allows each evaluator to have its own criteria set and also its own evaluation scale. Thus, it solves a relevant issue, performing the categorization even if the evaluators have different perceptions in terms of the composition of the criteria set.
• It allows each evaluator to have its own evaluation scale for the evaluation of the alternatives. Thus, it reduces the problem complexity once it avoids the need to develop a sole scale that should be understandable for each of the evaluators.
• It avoids the usage of compensatory procedure, such as the arithmetical mean, to generate inputs for a non-compensatory algorithm, as one of the previous ELECTRE methods.
Despite the simplification assumptions made while building the examples, the comparison of the application of ELECTRE TRI and ELECTRE TRI ME to the same problem has shown relevant differences in the results. These differences were caused by compensatory effects while adopting the arithmetical mean for generating the inputs to ELECTRE TRI. The adoption of ELECTRE TRI ME has avoided such undesirable effects.
Even though the examples are based on ELECTRE TRI ME, the ELECTRE ME proposal is general and can be extended to the ELECTRE family of methods as a whole. As a matter of fact, it should be extended to all multicriteria methods that are based on a table-grid, such as the Promethee family of methods.