Application of Various Multiple Criteria Analysis Methods for the Evaluation of Rural Road Projects

2.1. Methodology

This paper extends previous works by Bhandari, Shahi, & Shrestha (2014; 2016) [19, 20] in the multi-criteria evaluation of rural transportation projects in the case of Nepal. This study in the application of multiple criteria analysis methods for the evaluation of rural transportation projects was conducted with the determination of criteria, weighting of criteria, measurement of criteria, scaling of criteria, and aggregation of the performance of criteria with three multi-criteria analysis methods (viz. TOPSIS, MOORA, and PROMETHEE). Four rural roads from the Dang district of Nepal were evaluated. Dang district is an inner Terai district, located at about 280 km west of Kathmandu. Dang covers an area of 2955 km² with a population of 552,583 inhabitants. The road network in the district is comprised of 807.11 km. Dang district has 24 district roads and 16 village roads. These roads are mainly earthen and gravel type. The local authority has to upgrade these roads to all-weather road standards. Due to the limited budget for the four rural roads of the district road category, that belong to different geographical locations with the characteristics given in Table 1 the interventions have to be prioritized.

2.2. Determination of Criteria

Evaluation criteria are important elements for ranking rural road projects, for which there are scarce resources. The evaluation criteria involving 3 aspects of sustainability for ranking rural road projects were determined in this study. The criteria for sustainability assessment depend significantly on the project stakeholders understanding of sustainable development [21]. The evaluation criteria for this study were derived from a thorough literature review, a personnel interview, and an online survey among different experts. A pilot survey was conducted among seven Nepalese experts working in the field of rural roads. This study identified 13 sub-criteria under the headings of the three pillars of sustainability: economic, social, and environmental. The sub-criteria were validated with a wider survey of experts from 24 countries and Nepal through an online questionnaire. The identified criteria and sub-criteria are presented below in Fig. (1).

The brief description of the above mentioned sub-criteria and their measure are given below in Table 2.

2.3. Determination of Weight

The relative importance or weight of the criteria is a key step in a multiple criteria evaluation. The following methods can be used to establish the weights: a) equal weighting, b) direct weighting, c) regression based observer derived approach, d) the Delphi method [22], e) pairwise comparisons or AHP, and f) Likert scale. The last two methods, AHP and Likert scale, were applied by the first author for the determination of the weights [19, 20] due to the robustness and simplicity of these two methods. The same weights were applied in this study.

The AHP model is based on a basic set of the four axioms [23]. The first axiom states that given any two evaluation elements, stakeholders must be able to provide a pair-wise comparison. The second axiom requires that when comparing any two elements, stakeholders should never decide that one indicator is infinitely superior to another. The third axiom states that the evaluation must be formulated as a hierarchy [23]. In AHP, the criteria are synthesised to a different level of the hierarchy. The overall objective of the decision is at the top level. The criteria used in arriving at this decision are in the lower level. Three major criteria are taken, i.e. costs (economic), social, and environmental aspects in the second level [19]. In the third level economic costs are synthesised into financial and social costs. The financial cost, social costs, social aspects, and environmental aspects have three, three, four, and three sub-criteria respectively. The decision hierarchy for multi-criteria evaluation is given below in Fig. (2).

The criteria of each level were compared pairwise in order to determine the relative weights of all criteria & sub-criteria and a pairwise comparison matrix was developed as suggested by Thomas L. Saaty [24]. Saaty suggested an arbitrary rating scale of 1 to 9 based on psychological experiments. The definition of each scale is as follows:

• 1: Two criteria are equally important
• 3: Moderate importance of criterion X over criterion Y
• 5: Strong importance of criterion X over criterion Y
• 7: Very strong importance of criterion X over criterion Y
• 9: Extreme importance of criterion X over Y

2, 4, 6 & 8: Intermediate values between the values mentioned above

With this rating values as elements of matrix a_ij of the criteria i and j (where a_ij are the geometric mean values of responses), a pair-wise comparison matrix A has been developed as follows:

(1)

A Likert Scale is a psychometric scale commonly used in questionnaire surveys. It is simple to construct and easy to be read and filled by the participants. The Likert scale is the sum of responses to several Likert items. A Likert item is simply a statement, which the respondent is asked to evaluate according to any kind of subjective or objective criteria; generally, the level of agreement or disagreement is measured. In this research, an 11-point Likert scale is used to evaluate the weightage of evaluation criteria for rural transportation projects.

The questionnaire for both methods was set up on an online survey tool (Google Forms). Table 3 shows two example questions from the questionnaire in Likert Scale.

2.4. TOPSIS Method

Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) is “an approach that originates from the geometric concept of the displaced ideal point, according to which a criterion under investigation is seen to be situated in relation to its ideal positive (most favourable) and negative (least favourable) locations” [14]. This method considers three types of attributes or criteria: qualitative benefit attributes/criteria, quantitative benefit criteria, and cost. In this method, two alternatives are hypothesised; i.e. ideal and negative ideal. The ideal alternative has the best level for all attributes considered and the negative alternative has the worst alternative value. TOPSIS selects the alternative, which is the closest to the ideal solution and farthest from negative ideal solution. For the decision making with TOPSIS method, there should be m alternatives and n criteria and a score of each option with respect to each criterion. The following steps are taken with TOPSIS method for decision making [25]:

(i) Step 1: Construct the Normalized Decision Matrix (r_ij):

(2)

Where x_ij = evaluation parameters.

(ii) Step 2: Construct the weighted normalised decision matrix (v_ij):

(3)

Where (w_j)=weightage of parameters

(iii) Step 3: Determine the ideal and negative ideal solutions:

Ideal solution (A*):

(4a)

Negative ideal solution (A'):

(4b)

(iv) Step 4: Calculate the Separation Measures for each alternative:

The separation from the ideal alternative (S^*_i) is:

(5a)

Similarly, the Separation Measure from the negative ideal alternative (S^'_i) is:

(5b)

(v) Step 5: Calculate the Relative Closeness to the Ideal Solution C^*_i:

(6)

Select the option with C^*_i closest to 1.

2.5. MOORA Method

Multi-Objective Optimisation on the basis of Ratio Analysis (MOORA) method was first introduced by Brauers and Zavadskas (2006) [15]. MOORA method is composed of five steps [15, 16, 26-28]:

(i) Step 1: The method starts with a matrix of responses of different alternatives on a different objective (X):

(7)

where, x_ij x–the response of alternative j on objective or attributes i;

i = 1, 2…. n i=1,2…. n objectives or attributes;

j = 1, 2…. m:j = 1,2……m: the number of alternatives.

(ii) Step 2: The second step is the normalisation of a matrix of responses with the ratio system. Ratio System: In MOORA method each response of an objective is compared to a denominator of representative for all alternatives concerning that objective (x^'_ij).

(8)

Thus, normalised responses of the alternatives on the objectives belong to the interval [0,1][0,1].

(iii) Step 3: The third step is an evaluation of positive and negative effects (x^'_ij). For optimisation, these responses are added in case of maximisation and subtracted in the case of minimisation:

(9)

where: i = 1,2,….g_i as the objectives to be maximise di = g + 1, g+ 2,….n as the objectives to be minimised.

(iv) Step 4: In the fourth step, weighted assessment value is determined. In order to give an importance to the criteria, they could be multiplied by the importance factor of the criteria [15].

Finally, all alternatives are ranked, according to the obtained ratios (i.e. decreasing value of y^*_j).

2.5. PROMETHEE Method

Preference Ranking Organisation Method for Enrichment Evaluations (PROMETHEE) belongs to the family of outranking methods as it was initiated by B. Roy at the end of the ‘60 with the ELECTRE methods [29]. The original PROMETHEE methods were conceived by J. P. Brans in 1982 and were further extended by Vincke and Brans in 1985 [18]. The PROMETHEE methods include several unique tools for this purpose, such as the weight stability intervals and the Walking Weights interactive procedure. In 1989, the introduction of Geometrical Analysis for Interactive Aid (GAIA) added a descriptive complement to the PROMETHEE rankings. A graphical representation of the multi-criteria problem enables the decision maker to better understand the available choices and the necessary compromises he or she will have to make to achieve the best decision. GAIA can also be used to see the impact of the criteria weights on the PROMETHEE rankings. “The evaluation table is the starting point of the PROMETHEE method. In this table, the alternatives are evaluated on different criteria. These evaluations involve essentially numerical data” [30]. There are several types of PROMETHEE (viz. PROMETHEE-I, PROMETHEE-II, PROMETHEE-III, PROMETHEE-IV, PROMETHEE-TRI, PROMETHEE CLUSTER, etc.). The most common used methodology is PROMETHEE-II, which is used in complete ranking. “It is interactive and is able to classify and order alternatives which are complex and difficult to compare. In addition, it has other characteristics such as: simplicity, clarity and stability” [31]. The basic principle of PROMETHEE II is based on a pair-wise comparison of alternatives along each recognised criterion. Alternatives are evaluated according to different criteria, which have to be maximised or minimised.

The implementation of PROMETHEE-II requires two additional types of information, namely: information on the relative importance (i.e. the weights) of the criteria considered and information on the decision maker’s preference function, which he/she uses when comparing the contribution of the alternatives in terms of each separate criterion. The weights and the preference function determine the preference structure of the decision-maker. PROMETHEE does not provide specific guidelines for determining these weights but assumes that the decision-maker is able to weigh the criteria appropriately, at least when the number of criteria is not too large. The preference function (P_j) translates the differences between the evaluations (i.e. scores) obtained by two alternatives (a and b) in terms of a particular criterion, into a preference degree ranging from 0 to 1. The procedure for implementing PROMETHEE-II “is started to determine deviations based on the pairwise comparison. It is followed by using the relevant preference function for each criterion in Step 2, calculating [the] global preference index in Step 3, and calculating positive and negative outranking flows [strength & weakness] for each alternative and partial ranking in Step 4” [18]. Finally, the procedure ends in Step 5 with the calculation of net outranking flow for each alternative and complete ranking.

² Another approach would be to calculate the Cost/Benefit (C/B) ratio or else the Economic Internal rate of Return (EIRR) for the economic pillar combining the first six parameters into a single indicator. Moreover, an even better approach could be using the indicator Net Present Value (NPV)/Construction Cost, which normalizes the net benefits in relation to cost. Since the parameters 11, 12 and 13 have the same ordinal scale (1-9) and they refer to the environmental pillar, these could be combined into a composite index. The measurement of parameters 7, 8 and 9 are explained in Table 1. Moreover, for the parameters 7, 8 and 9, the same discrete 5-point scale that was used for the parameter 10 could be used too. With such an approach, the array of factors could be streamlined and hence possible double-counting of benefits would be avoided, however, the above-mentioned 13 indicators are used in this way in this study to make the different aspects of rural roads more understandable to the decision makers.

3.1. Ranking with TOPSIS Method

The strength of TOPSIS is that it is easy to implement having an understandable principle, it is applicable when exact and total information is collected, it considers both the positive and negative ideal solutions, and it provides a well-structured analytical framework for alternatives ranking [7]. The ranking of the abovementioned roads with TOPSIS is described below.

1. Step 1: The decision-making was formulated with the evaluation parameter x_ij as given above in Table 4.
2. Step 2: The Normalised Matrix r_ij of the four roads is presented below in Table 5.

(iii) Step 3: The Weighted Normalised Matrix is calculated by the multiplication of the Normalised Matrix [r_ij] with the weights of the respective criteria in AHP as specified in Table 4. The results with AHP are presented below in Table 6.

(iv) Step 4: The Ideal Solution and the Negative Ideal Solution of the Weighted Normalised Matrix are presented below in Table 7.

(v) Step 5: The separation measures from the ideal solution and the negative ideal solution of each road are presented below in Table 8.

(vi) Step 6: The Relative Closeness and their ranking order of the rural transportation projects are presented below in Table 9. The ranking of the rural road projects is as per the decreasing order of the Relative Closeness of the respective roads.

3.2. Ranking with Multi-Objective Optimisation on the Basis of Ratio Analysis (MOORA) Method

The same Decision Matrix with the TOPSIS is used as presented in Table 4. Then, the Ratio Matrix is derived as presented in Table 10. Similarly, the Normalised Matrix is presented in Table 11.

Finally, the ranks are determined based on the Reference Point and the Deviation from the Reference Point as presented in Table 12.

3.3. Ranking with PROMETHEE Method

The ranking of the above-mentioned roads was also studied with the application of the VISUAL PROMETHEE MCDA decision aid software. The academic version of this software was used in this case study. The software was downloaded from the following link http://www.promethee-gaia.net. The same input of impact matrix and the weights derived with AHP presented in Table 4 were used in this study. All criteria were defined by a linear preference function. The statistics of the data of the rural road projects are presented in Table 13. The preference parameters and rankings with PROMETHEE are presented in Tables 14 and 15, respectively. The Range of Stability derived from Visual Stability Interval at Stability Level four is presented in Table 16. The results are presented in Figs. (3-7).

In Fig. (3) the partial ranking of the analysis with PROMETHEE-I is presented. It can be noticed that the Road B is the highest and the Road C is the second highest preferred to all the other roads and that Road D is the lowest preferred to all the others road projects in PROMETHEE-I partial ranking. No projects are identified as incomparable (i.e. worse phi- and best phi+ or reverse while comparing between two projects).

In Fig. (4) the PROMETHEE-II complete ranking shows that Road B has the highest phi. Road C has the second highest phi. Road A and Road D have almost similar net phi. Road D has the lowest net phi value. In Fig. (5) the PROMETHEE Diamond shows that Road B (in red colour) overlaps all the other roads. Obviously, this project is preferred to all the other projects according to the PROMETHEE-II partial ranking.

Fig. (6) represents the GAIA Plane obtained principle component analysis to the alternative of the problem. At first, it may be noticed that the alternative Road C performs well on the environmental, maintenance cost, and pollution cost criteria. It is neutral in travel time saving cost and accident cost criteria and it has a poor performance on the population served per km, vehicle operation cost saving, access to educational services, and access to other services criteria. On the other hand, Road B performs well on access to educational services, access to other services, population served per km, and vehicle operation cost saving criteria. It is neutral in the travel time savings cost, construction cost, and accident cost criteria and it has a poor performance on maintenance cost and impact on natural resources criteria. On the contrary, Road D performs well on the construction cost and travel time saving cost criteria. It is neutral in access to educational services, access to other services, population served per km, vehicle operation cost saving, impact on natural resources, and pollution cost criteria and has a poor performance on accident cost criterion. Similarly, Road A performs well on accident cost criterion. It is neutral in vehicle operation cost saving, population served per km, pollution cost, impact on natural resources, and maintenance cost criteria and it has a poor performance on travel time saving cost and construction cost criteria. Thus, the use of complementary tools such as the GAIA Plane, Diamond, etc. provides support to the decision maker in order to understand the result and make the best selection for the final solution.

In Fig. (7) the positive and negative outranking flows show that Road B is preferred compared to all the other roads. Road C and Road A follow. Road D is the least preferred alternative. No road projects are incomparable.