A Novel Approach for Systematic Literature Reviews Using Multi-Criteria Decision Analysis

Modern discussions of MCDA methods date back about half a century [1]. MCDA is often used by governments and cities to support rigorous decision-making [2], and and these methods are widely used by researchers to assist in decision-making in different areas of knowledge [3]. Since it was launched in the mid-1960s, MCDA has become important and demanded to be a useful tool due to its wide applications in a number of practical problems [4].

Different MCDA techniques were created, improved/adapted and used along with other methods over time, with the aim of generating results with increasingly robust methodologies and enabling their wider application. Overall, these methods are divided into compensatory and non-compensatory methods [5], [6]. Hwang and Yoon [5] mention that in compensatory methods it is possible to perform the compensation between criteria, while in non-compensatory ones this compensation is not possible.

Hwang and Yoon [5] mention that in compensatory models it is possible to compensate between criteria, while in non-compensatory models this compensation is not possible. Maracajá et al. [7] concluded that compensatory methods are those that include all the processes of the additive model, compensating the poor performance of an alternative in a given criterion for a good performance in another criterion. In theory, this is known as a trade-off, where the decision-maker “gives up” the performance of the alternative in one criterion to “accept” it due to the good performance that was compensated in another criterion.

The different approaches (compensatory and non-compensatory) provide researchers with a range of methods that can be used in studies. Some of the methods that are widely used, with their forms of approaches, are: Simple Additive Weighting (SAW) [5], Technique for TOPSIS [5] and AHP [8], are some methods that can be classified as compensatory, while the family ELimination and Choice Translating Reality (ELECTRE) methods, proposed by Roy [9], and the family Preference Ranking Organization Method for Enrichment Evaluations (PROMETHEE) methods, proposed by Jean-Pierre [10], are classified as non-compensatory methods [6].

In recent decades, authors from different areas have used these methods in various applications. Many methods have been created or adapted, creating new possibilities for methods to be considered for application to practical problems. Therefore, it is important to check which methods have been most applied in recent years, where they are being applied, among other issues.

It is important to analyze publications involving compensatory and non-compensatory methods in MCDA. Researchers have performed SLR involving MCDA in relation to different themes: Failure mode and affect analysis [11], solution rural land allocation problems [12], renewable energy site selection [13], passive energy consumption optimization strategy selection for buildings [14], E-learning [15], supplier selection [16], multi-species building envelopes [17], optimizing CO$_2$ decisions in the automotive industry [18], measuring circular economy [19], achievement of the UN sustainable development goals [3], sustainable selection of insulation materials in buildings [20], financial modeling [21], Urban Freight Distribution [22], legacy system modernization [23], risk management [24], Solar Power Plant Site Selection [25], corporate sustainability [26] and sustainability engineering [27].

Applications of specific MCDA methods have also been analyzed through SLR. Kaya et al. [28] reviewed the applications of fuzzy MCDA methodologies applied to energy policy making. The results showed that fuzzy AHP, whether applied individually or combined with another technique, was the most applied and that type 1 fuzzy sets are the most preferred types of fuzzy sets. Dos Santos et al. [29] developed a study on the applications of the AHP method in sustainable development, with the results showing that the fuzzy AHP technique was the favorite technique to support sustainable decisions and having the main applications in manufacturing and urban-related sustainability.

Paul et al. [30] performed an SLR on the application of fuzzy MCDA in service selection, analyzing publications from 2010 up to 2021. A total of 508 publications were found, considering 60 articles after applying the PRISMA method, proposed by Liberati et al. [31]. The most used integrated method was fuzzy AHP plus fuzzy TOPSIS.

Aires and Ferreira [32] explored the research regarding the reverse rank in MCDA problems, considering 130 articles published in international journals related to the subject between 1980 and 2015. The most explored method in the problem of ranking reversals was the AHP, and the addition and/or removal of irrelevant alternatives was the most considered analysis criterion.

A search in the literature pointed to a gap in the creation of the portfolio with the use of incorporated MCDA techniques. There are SLR methods used to sort papers in order of importance, like Methodi Ordinatio [33] and NIRP [34], but in these methods there are some parameters that are chosen by the researcher in a subjective way. Therefore, the objectives of this article are:

- The adaptation to the SLR method proposed by Steffen et al. [34], with a methodological advance in relation to the ranking criterion;

- A review of applications of compensatory and non-compensatory methods in articles involving MCDA, aiming to determine the main areas of application and gaps in the literature.

In this case, for the first objective, this SLR method and the MCDA methods will be combined to calculate the weights and create the ranking of papers. The calculation of criteria weights will be performed considering the AHP and EWM and the ranking of publications will be created with the TOPSIS and an adaptation to the index pattern by Steffen et al. [34], which considered the use of the AHP and TOPSIS methods to construct the ranking of articles, with a view to compensating the criteria. For comparison purposes, since the same criteria as the aforementioned study are used, this analysis also considered the calculation of weights using the EWM method. The AHP method considers subjectivity for the judgment matrix, while the EWM method provides a different analysis in calculating weights, directly targeting the criteria values. This comparison makes the calculation of weights more robust and a small methodological advance in relation to the study by Oliveira et al. [35], with the weight values presenting similar results, despite the difference between the methods.

The second objective seeks to build a portfolio involving compensatory and non-compensatory methods, aiming to find their approaches and forms of application in data and also the main areas of application, among other aspects.

Research in the literature pointed to a gap in the creation of a portfolio of papers using MCDA techniques in the ranking procedure. Also, none of the SLR studies involving MCDA have recently performed a comparison between compensatory and non-compensatory approaches. This possibility provides an analysis of the most commonly used methods and an insight into new approaches, in addition to the two conventional ones, that have emerged in recent years.

This article is divided into sections. The methodology section presents the development of this study. The results section gives an analysis about the development of the SLR and a bibliometric and systematic analysis of the final portfolio. Finally, the conclusions and directions for future studies are presented in the last section.

SLR is a method for locating articles in a given area or thematic, describing more utilized methods, main applications, and determining researched crucial points. The main method of synthesis on a specific topic or research question, the SLR is a rigorous methodological review of the literature, as it is not only a way to join all existing evidence on a topic, but a way to support the development of guidelines based on evidence [36].

This study developed an adaptation to the methodology proposed by Steffen et al. [34], whose steps are described in Figure 1, supported by MCDA methods and creating a hybrid method in the search for defining the criteria weights and creating a portfolio of publications, based on the ranking of publications. This new methodology was applied in the search for articles about “compensatory and non-compensatory" methods in “MCDA".

Steps 1-3: In the initial stages, authors must define the research intentions, the databases that will be considered, and carry out exploratory research to define keywords. Step 4 consists of the final search in the selected databases, creating the initial portfolio.

Step 5 is focused on filtering the articles. There is a possibility that some articles may appear more than once when searching in different databases and using different combinations of keywords. Therefore, it is necessary to eliminate the duplicates. This deletion can be carried out using software (such as Endnote, Mendeley, JabRef, etc.) or manually. The next filtering procedure is elimination by reading the title, keywords, and abstract, which help us verify whether the article really addresses the subject we want to analyze. In step 7, publications from journals without impact or older than 5 years without citations were excluded. To ensure the consistency and reliability of the results, the filtering and indicator removal procedures were carried out by two researchers and compromised at least twice by each of them. This achievement is indicated in SLR articles, as indicated by Pagani et al. [37].

The change in the development of this new methodology is directly linked to step 8 of the aforementioned method, as shown in Figure 2. In this step, MCDA methods are incorporated into the analysis.

In the process of determining criteria weights for ranking the portfolio, the AHP, as proposed in references [8], [38], [39], [40], [41], [42] and the EWM, proposed by Shannon [43], were utilized. The weights for each criterion were calculated by averaging the results from both the AHP and EWM methods. The ranking of articles was then conducted using the TOPSIS, developed by Hwang and Yoon [5], along with an adaptation of the equation proposed by Steffen et al. [34].

For the creation of the final ranking, an arithmetic mean of the values found for each ranking, proposed by Steffen et al. [34] and the TOPSIS method, generating a final index (I), according to Eq. (1), and based on the work of Oliveira et al. [35]. For both methods, a conversion of values was performed (1 for the maximum value and the others calculated proportionally). Calculating the weights and creating the ranking from two methods for each case makes it easy to visualize possible discrepancies and limitations caused by one of the methods. Furthermore, the use of methods in this study is based on the compensatory nature of the criteria.

The $D_i^*$ is obtained by the TOPSIS method, and the $Y_i$ index is an adaptation of the equation proposed by Steffen et al. [34] as described by Eq. (2):

where, $K_{i1}$, $K_{i2}$ and $K_{i3}$ are the normalized values of $\overline{K}_{i1}$, $\overline{K}_{i2}$ and $\overline{K}_{i3}$

$\overline{K}_{i1}=\frac{C_i}{(Age)_i+1}$, $\overline{K}_{i2}=\frac{(IF_k)_i}{n}$ and $\overline{K}_{i3}=(self-citations)_i$, $i=1,..,n$

$C_i$ = number of citations of alternative $i$ (collected on Google Scholar), according to previous studies [33], [35], [37], [44], [45];

$(Age)_i$ = age of the article $i$ (up to the year the search was carried out);

$(IF_k)_i$ = Impact Factor $k$ (SNIP, JCR, SJR,...) of the journal $i$ ;

$(Self-citations)_i$ = index of the journal $i$ related to the number of citations of a journal by articles published in that journal (found at: https://www.journalindicators.com/indicators); and, $w_j, j = 1,2,3$ are the weights considered for each criteria.

Research for the development of the SLR was carried out in three databases: Scopus, ScienceDirect, and Web of Science [19], [44], [45]. The MCDA methods are described in the next subsections.

The criteria weights calculated using the AHP method, proposed by Saaty, were carried out by creating a judgment matrix of the A criteria (with $a_{ij}$ elements). Each matrix element represents the degree of importance of a criterion in relation to another criterion.

$\mathbf{A}=\left(\begin{array}{cccc}1 & a_{12} & \ldots & a_{12} \\ \frac{1}{a_{12}} & 1 & \ldots & a_{2 n} \\ \vdots & \vdots & 1 & \vdots \\ \frac{1}{a_{1 n}} & \frac{1}{a_{2 n}} & \ldots & 1\end{array}\right)$

where,

i) $a_{ij}>0$

ii) $a_{j i}=\frac{1}{a_{ij}},~ reciprocal$

iii) $a_{i i} = 1$

iv) $a_{ik} = a_{ij}\cdot a_{jk}$

This matrix being filled according to the scale of the study by Saaty [41], presents in Table 1:

From the matrix of judgments, a new normalized matrix B (composed of $b_{ij}$ elements) is calculated [8]:

and priority eigenvector ($W_i$):

and

Eigenvector represents the value of the matrix elements in relation to the total. To verify the consistency of the calculations, Saaty [41] created a consistency index (CI), and this index cannot be greater than 10$\%$. For this calculation, first multiply the elements of matrix A by their respective weights ($W_i$), creating a new matrix C ($c_{ij}$ elements):

Add each line of C and divide the value by its respective weight $W_i$, generating an $H_i$, value, and calculate the eigenvalue associated with this matrix ($\lambda_{max}$), which is the arithmetic mean of $H_i$:

Consistence index (CI) is calculated by

where, RI (random index), proposed by Saaty [41], is described in Table 2:

To deal with possible limitations and discrepancies of the AHP method, the EWM method was also considered. It is important to highlight that despite the calculation of the consistency index, the judgment matrix is constructed only from the subjective view of the decision-maker.

The EWM is a method proposed by Shannon [43] that is used in MCDA to determine the weights of each exam considered in the analysis. Create the matrix R (with $r_{ij}$ elements):

$\mathbf{R}=\left(\begin{array}{cccc}r_{11} & r_{12} & \ldots & r_{1 n} \\ r_{21} & r_{22} & \ldots & r_{2 n} \\ \vdots & \vdots & \vdots & \vdots \\ r_{m 1} & r_{m 2} & \ldots & r_{m n}\end{array}\right)$

where, $R$ describes the decision matrix of the $m$ alternatives in relation to the $n$ criteria. Before carrying out the weighting according to Shannon [43], a normalization procedure was performed, as was performed by Kumar et al. [46], aiming to make the data to be on the same scale [0-1] and all standardized with the idea that the higher the better. The method of maximums and minimums, proposed by Chakraborty and Yeh [47], generates a new matrix X (with elements $x_{ij}$):

The use of normalization means that all data appear on a bigger-better scale, standardizing the data. Furthermore, this normalization method can assist in possible surveys incurred in creating the ranking using the TOPSIS method, which can present ranking reverse, according to Aires and Ferreira [32]. After that, the weighting of the data is considered, as explained above, by dividing the $x_{ij}$ values by the sum of column j, and the calculated value of entropy ($e_{j}$):

and the degree of diversification ($d_{j}$):

The calculation of the weights for each criterion ($EW_{j}$):

After calculating the weights, the ranking of the articles found was determined considering the TOPSIS method and an adaptation to the index proposed by Steffen et al. [34], as presented in Eq. (2). For the case of values of $x_{ij}$ equal to 0, thse values of $x_{ij}\,log(x_{ij})$ were also considered equal to 0 for the calculation described in Eq. (11). Therefore, these values had no impact on the results.

The TOPSIS method was proposed by Hwang and Yoon [5]. The application of the TOPSIS method was carried out as proposed by Aires and Ferreira [32], considering the normalization of maximums and minimums [47]. Therefore, the calculations started with the matrix X.

The next step of the method is the multiplication of the weights of each criterion (which in this case was considered by the average between the weights of the AHP and entropy methods) by the elements of matrix X, generating a new matrix V (with elements $v_{ij}$).

$\mathbf{V}=\left(\begin{array}{cccc}v_{11} & v_{12} & \ldots & v_{1 n} \\ v_{21} & v_{22} & \ldots & v_{2 n} \\ \vdots & \vdots & \vdots & \vdots \\ v_{m 1} & v_{m 2} & \ldots & v_{m n}\end{array}\right)$

After that, the ideal and the negative-ideal solution are considered, generating the alternatives $A^*$ e $A^-$:

where,

Hwang and Yoon [5] mentioned that the two alternatives ($A^*$ and $A^-$) will indicate, respectively, the most preferable alternative (ideal solution) and the least preferable alternative (negative-ideal solution). The calculation of the distances of each alternative m, ideal and negative-ideal, can be measured by the n-dimensional Euclidean distance, presented, respectively:

The calculation of the ideal solution is performed:

From the value $D_{i}^*$ it is possible to create the ranking of the alternatives in descending order, that is, the highest value indicates a better performance of the alternatives in relation to the criteria.

For the purposes of comparisons and possible discrepancies in the ranking results, ranking calculations were carried out considering the two methods reported in this study (TOPSIS and an adaptation of Steffen et al. [34]). Furthermore, the normalization method proposed by Chakraborty and Yeh [47] to deal with possible limitations of the TOPSIS method, such as ranking reversal, was analyzed by Aires and Ferreira [32].

The research intention was defined as exploring compensatory and non-compensatory methods in MCDA and the three databases that were considered (Scopus, Web of Science, and ScienceDirect). Exploratory research helped define the keywords considered, which are presented together with the results found in each database in Table 3.

A total of 267 articles were found. The next steps are related to the filtering procedures, shown in Table 4.

After the initial filtering procedures (Step 5), searches were carried out by the considered indicators (criteria) and a second round of filtering (Step 7) was carried out. With this, the analysis was performed considering 96 articles. Step 8 of the analysis started with the construction of the weights, using the AHP and EWM methods. The AHP method starts with the criteria judgment matrix, shown in Table 5.

From the matrix of judgments, the weights for the criterion are determined, and the CI is verified. The CI was 0.088, which is in line with what was proposed by Saaty [41]. The criteria weights for the AHP method and for the EWM method (created from the normalized matrix, with the method of maximum and minimum, of the indicators), in addition to the average of the weights considering these two methods, are presented in Table 6.

The calculation of the weights made it possible to create the ranking of the articles, considering the average of the weights calculated by the methods. It is worth mentioning that two criteria are considered benefit (number of citations and impact factor), while one criterion is considered cost (self-citations). The number of citations represents how many times a publication was cited by other articles in the area and the impact factor represents a measure of the impact of a journal in a given area. Thus, for this type of data the bigger the better. On the other hand, self-citation represents a percentage of the number of times a journal was cited (considering all its articles) by its own articles. Therefore, the bigger the worse. The ranking was created for the TOPSIS method (R01), through an adaptation of the one proposed by Steffen et al. [34], R02, and the average between these (R03), presented in Table 7. The final portfolio consisted of 90 articles, since it was not possible to download the complete 6 articles.

As previously mentioned, no publications were found in the literature involving the construction of a portfolio of articles using MCDA methods for ranking the papers. This analysis allows the results presented in the construction of the portfolio to go according to the indicators considered, considering that the rankings were created with the values determined by criteria that belong to the own articles and to the journals where they were published. The next section will present a bibliometric analysis of the final portfolio.

Figure 3 displays the analysis of publications based on the countries of the institutions affiliated with the first authors at the time the papers were published in the final portfolio. Brazil leads with 18 articles, accounting for 20$\%$ of the total. of the total. France and Iran follow, contributing 11 and 10 articles, which represent approximately 12.22$\%$ and 11.11$\%$ of the portfolio, respectively. China, India, and Italy each have contributed 5 publications, while Portugal has 4 articles. Australia, Spain, Taiwan, and Turkey each contributed 3 articles. Additionally, Canada, the United States, and Sweden are each represented by 2 articles. A further 12 countries—Belgium, Chile, Greece, Ireland, Netherlands, Nigeria, Pakistan, Poland, Serbia, South Africa, Switzerland, and the United Kingdom—each have one article. The association of a publication with a country and institution was determined based on the first author’s affiliation.

In addition to the countries, a search was carried out for the universities/companies that participated in the portfolio (Figure 4), allowing, through this, to indicate possible existing research groups.

The main institution is the Federal University of Pernambuco, with 8 papers. Fluminense Federal University and University of Tehran, have 4 publications each, Central South University Islamic Azad University, have 3 articles each, and Central South University, Université Paris 8, Université Paris-Saclay, University of Coimbra and University of São Paulo, have 2 papers each, were also universities/companies that contributed more than 1 publication. Another 58 institutions contributed with 1 publication each, making a total of 68 institutions. Particularly, the Federal University of Pernambuco and Fluminense Federal University, both from Brazil, have strong research lines involving MCDA, which may explain the fact that they are well placed in terms of the number of publications involving universities.

Regarding the time range, there was no time cut, that is, all publications found in the literature were considered. Figure 5 presents a temporal analysis of publications. 2020 was the year with the highest number of publications, about 14.44$\%$ of the articles, followed by 2021 and 2022.

Despite the initial stability, it is possible to verify an increase in relation to the first publication found, 1986, over time. In order to carry out a more assertive analysis in relation to the time period of the analyses, a graph was created where the data were grouped every 6 years (with the first group counting the 4 publications, from data collected up to March 30, 2023), as shown in Figure 6.

With this, the increase in publications for the last period is clear. The authors with the most contributions are shown in Figure 7.

Banihabib, M.E. and De Almeida, A.T. contributed with 4 publications each, being that Banihabib, M.E. has 3 publications as the first author and one as the second author and a total of 592 citations. De Almeida, A.T., on the other hand, has one publication as the first author and 3 as the second author and a total of 220 citations. Costa, H.G. and Lima Junior, F.R. have 3 publications each, while 18 other authors contributed 2 publications for the composition of the portfolio and 232 contributed 1 article each. Figure 8 presents the number of citations per year.

For the construction of this graph and its better visualization, the number of citations per year divided by 10 and the average of citations were considered. The average of citations considered in Eq. (21):

where, AC = average citations; CP = total number of citations up to the research date; YP = year in which the paper was published; and, NP = number of publications, from the final portfolio, in the year considered.

Through this, it is verified that the years 1998, 2015, 2018, 2013, and 2012 were the ones with higher values of citations, but when we consider the average number of citations in relation to the age of the articles, the years with the highest average citations are: 2014, 2013, 1998, 2009 and 1986. In 1986, for example, only one article was published and obtained 269 citations in total, which, on average, considering its age plus one, leads to an average number of approximately 7.08. Figure 9 presents the journals that appear most in the portfolio.

The European Journal of Operational Research (7 articles) is the journal with the greatest number of publications in the portfolio, representing approximately 7.78$\%$, followed by the journal Applied Soft Computing, with 6 publications (6.67$\%$). Another 3 journals (Annals of Operations Research, Brazilian Journal of Operations and Production Management and Water Resources Management) present 3 publications, while 6 contribute 2 publications each and 56 with 1 publication each. It is noticed that there are journals focused on different areas of portfolio formation. Figure 10, built with the VOSviewer software, presents the keywords that appear the most in the 90 articles that appear in the portfolio, considering the keywords that appear at least 2 times.

“Decision-making” is the keyword with the highest number of occurrences, present in 22 articles, interconnecting the different groups created. “decision-theory”, “multicriteria analysis”, “sustainable development” and “decision support systems” are keywords that appear with more occurrences, respectively, with 8, 7, 6 and 5 occurrences. It can be noted that words related to sustainability appear 17 times, indicating one of the possible topics most discussed in the articles. Keywords linked with the general term “environmental” appear 10 times, pointing to other possible study applications.

In this sense, a count of the most frequent words in the articles was carried out, making it possible to verify the main topics addressed, using the Nvivo 14 software, as shown in Figure 11. This analysis reinforces the words already highlighted in the previous item, reinforcing the main themes, which will be addressed more specifically through a systematic reading of the articles.

After systematic reading and analysis of the articles, it was possible to establish the form of approach presented, the most commonly used methods, among other aspects. Table 8 shows the approach used. It was found that, in addition to the conventional approaches, compensatory and non-compensatory, other forms emerged.

The majority of articles, in the traditional way, use compensatory or non-compensatory methods. 40$\%$ of the articles mention the use of compensatory data, while approximately 26.67$\%$ use the non-compensatory form, the same percentage that uses compensatory and non-compensatory methods together in the same publication. Different approaches are also present in the articles (compensatory, non-compensatory, and partially compensatory, semi-compensatory, semi-/non-compensatory, partially non-compensatory and compensatory and non-totally compensatory).

The most commonly used methods are presented in Figure 12. In this case, it was considered whether the methods were present in the analysis, either individually, combined with another method, or generating a hybrid method. In addition, the methods were considered in relation to their family of methods, when applicable.

The methods of the ELECTRE Family are the most frequent, present in 29 articles [6], [53], [59], [61], [65], [67], [68], [74], [80], [82], [84], [88], [90], [98], [99], [101], [102], [108], [111], [116], [117], [126], [128]. Next, the AHP method appears, with 21 occurrences [6], [49], [50], [51], [52], [55], [62], [64], [66], [67], [71], [77], [84], [90], [106], [121], [123], [127], [129], [134], [135]. In the third place, the fuzzy method [55], [58], [63], [72], [77], [87], [89], [90], [91], [96], [101], [114], [122], [124], [127], [129], [134] and TOPSIS method [50], [51], [55], [59], [67], [80], [84], [90], [97], [103], [104], [108], [118], [125], [127], [129], [134], with 17 occurrences each. In addition to those mentioned, the methods of the PROMETHEE family stand out with 9 occurrences [52], [77], [88], [89], [93], [94], [97], [115], [133]. None of the other methods used in the articles, in addition to those mentioned, presented 5 occurrences or more.

Tukino et al. [137] performed an SLR involving MCDA methods for ranking alternatives, determining the weight of criteria and applications in the multi-criteria process. The results indicated that the TOPSIS, AHP and PROMETHEE methods are the most used for studies involving this approach. Oliveira et al. [35] checked publications involving MCDA and electric vehicles. Fuzzy, AHP and TOPSIS were the three most used methods for solving problems on the topics covered in these studies. The methods from the PROMETHEE and ELECTRE families were also among the most used. Maček et al. [138] carried out an analysis of publications involving MCDA methods for assessing information security risks. The results showed that the most used methods are, respectively: TOPSIS, AHP, PROMETHEE and ELECTRE.

In addition, the review also made it possible to verify the areas with the most occurrences of applications in the articles, shown in Figure 13.

The applications are quite diverse in the studies found, including theoretical articles, whose main focus is the development of a new method and the application occurs in generic data. The main areas of application of studies involving MCDA methods, compensatory and non-compensatory, are: sustainability, management, supplier selection, water and assessment. The search for sustainable solutions applying MCDM methods can be highlighted, as they are increasingly in focus and are evidenced by the results of the portfolio of articles found in this study, also evidenced by other authors [27], [35]. Even in other matters, such as supplier selection, which is also the target of many applications of multi-criteria methods, they bring approaches linked to sustainable issues [139].

This study allowed the construction of an SLR incorporating MCDA methods in order to sort the papers in order of importance, using the MCDA methods to determine the weights of each criteria used in the sorting process. According to our research, there are no studies carried out in this direction. With this, it was possible to create a ranking considering the criteria weights through the judgment matrix (AHP) and decision matrix (CRITIC), which makes the analysis more robust.

In addition, it was possible to map the studies with the keywords involving MCDA and compensatory, which resulted in the applications of MCDA and compensatory and non-compensatory approaches, in addition to different approaches, such as semi, partially and non-totally compensatory or non- compensatory methods.

It was verified that there are approaches that involve compensatory and non-compensatory in the same set of data, which theoretically would not be correct, since a set of criteria must present compensatory or non-compensatory. It was also noticed that non-compensatory methods were applied to compensatory data with the objective of eliminating the compensatory effects of the data.

In general, there is a difficulty in the literature to determine whether a set is compensatory or non-compensatory and the application of methods in relation to the correct nature of the data. Despite the limitations of these studies, which only generate results with the keywords used, these findings are well evidenced. Another restriction of the study may be linked to article exclusion methods applied by the proposed methodology. Conference articles or book chapters, for example, can provide different approaches that add to the portfolio. Therefore, other forms of SLR can be tested, and the results can be compared with those of this study.

For future studies, it is suggested to search for methodologies that seek to determine when a set of data is compensatory or non-compensatory for the correct application of methods. In addition, one can seek compensatory measures between the criteria using mathematical and/or statistical techniques.