All published articles of this journal are available on ScienceDirect.
Evaluating Sustainable Transport Indicators for Metropolitan Areas in Developing Countries: The Case of Greater Jakarta
Dwita Sutjiningsih2
Tri Tjahjono2
Linda Darmajanti3
Gede B. Suprayoga4
Authors Info & Affiliations
Abstract
Background:
Sustainable transport is fundamental to progress in realising the agenda of sustainable development, as a quarter of energy-related global greenhouse gas emissions come from the transport sector. In developing countries, metropolitan areas have adopted the agenda to better serve the urban population with safe, affordable, and environmentally-friendly transport systems. However, this drive must include relevant indicators and how their operationalisation can deal with institutional barriers, such as challenges to cross-sectoral coordination.
Objective:
This study aims to explore context-specific indicators for developing countries, focusing on the case of the Jakarta metropolitan area.
Methods:
Expert judgement was used to assess the selection criteria. The participants were experts from government institutions, non-government organisations, and universities.
Results:
The findings show that safety, public transport quality, transport cost, air pollution, and accessibility are contextual indicators for application in developing countries. Similarities are shown with the research results from other indexes/sets of indicators for developing countries, for example, the Sustainable Urban Transport Index (SUTI) of UN ESCAP. However, some of these indicators leave room for improvement, such as the balance between strategic and operational levels of application.
Conclusion:
Therefore, this research suggests that global sets of indicators should be adjusted before being implemented in particular developing country contexts.
1. INTRODUCTION
Sustainable development has become a global policy agenda in many countries since around 40 years ago. The main idea of this concept is that current development should not compromise the ability of future generations to fulfill their needs [1]. The concept incorporates environmental considerations into conventional development, in which economic and social aspects play a major role in development policies, programmes, and projects [2]. The implementation of the concept includes three aspects that should be considered simultaneously: environmental, social, and economical. In 2015, the United Nations launched its Sustainable Development Goals, based on three areas divided into seventeen goals and 169 targets, among which is Goal 11: Sustainable Cities and Communities. One of the targets of Goal 11 is by 2030 to provide safe, affordable, accessible, and sustainable transport systems for all citizens, including road safety improvement and public transport expansion, with special attention paid to vulnerable citizens. In addition, sustainable transport should support economic development and satisfy people’s needs while at the same time respecting natural laws and human values [3].
Transport is a complex system characterised by an interrelationship between economic, social, and environmental elements. Urban transport infrastructure facilitates the movement of people and logistics for different purposes. Urban inhabitants can take advantage of quality physical infrastructures that reduce travel time, expand economic opportunities, and intensify social interaction. However, transport can degrade the environment due to the emissions from engine combustion, noise pollution, and fossil fuel processing [4, 5]. Initiatives by the transport sector alone are inadequate to tackle such environmental problems [6]. Other sectors also need to contribute, such as those concerned with spatial planning, physical infrastructure, social interaction and institutional policy [7]. Therefore, a comprehensive perspective is required to ensure that all aspects of sustainable development are included in policy interventions for transport [8].
To be operational, the concept of sustainable transport development needs indicators, which are variables and units or scales used to measure the progress towards sustainability objectives. Indicators have several functions, such as explaining, illustrating, comparing, or quantifying the observed phenomena. They need to be specific and measurable clearly defining what is being measured. Shen and Zhou [9] identified at least five principal criteria for selecting an indicator: it should be specific in scope, coherent, inclusive, consistent, and should focus on the goals/vision. Several authors have studied sets of indicators to measure current trends and facilitate new initiatives' development. Sdoukopoulos et al. [10] established the distinguishing features of indicators from 78 studies and categorised them into dimensional and objective frameworks. Litman proposed indicators for sustainable transport usage in particular situations, ranked by priority and applicability [11, 12]. These were defined as Most Important (the ones normally employed), Helpful (used if possible), and Specialized (used to address particular needs or objectives). Jain and Tiwari [13] analysed the relationship between various indicators in a systematic approach to sustainable indicator selection using causal analysis that differentiated between root cause central and end-of-chain factors.
Distinct sets of indicators have been developed for actual use, such as ELASTIC [14], SPARTACUS [15], Green Apple [16], I_SUM [17], SUTI [18], and Lyon mobility [19]. However, these have yet to be evaluated for specific uses, for example, for urban or rural contextsor large or small cities. They are commonly used as a comparison tool, for example, for comparing performance between cities or countries. Therefore, the indicators have been developed for general contexts. However, the adoption of indicators needs to be context-specific to measure the phenomena to be understood [20]. Nevertheless, a comprehensive and holistic set of indicators is virtually impossible because of resource constraints in gathering and utilising such a broad set [21]. Therefore, indicators should be context-specific, aiming to measure what is relevant in a given space and time.
Jakarta metropolitan area is inhabited by 31 million people living in three provinces in this case study. Transport in the region is managed by a multitude of local and national organisations and sectors. The situation is different in western developed cities such as Halifax, Lyon, or Helsinki, which are inhabited by fewer than 2 million people and governed by single authorities. Therefore, adopting a set of indicators from international organisations may not effectively reflect the phenomenon to be measured. As an illustration, the Sustainable Urban Transport Index (SUTI) of UN ESCAP is implemented in Jakarta, with a score of 52.5 (the highest-ranking among three other Asian cities) [22]. However, the score may not reflect the usefulness and relevance of the ten indicator sets, since the data availability may influence their results and data selection choice of individual cities [22]. Therefore, we suggest that further refinement of the indicator sets and indices is necessary.
Application of the same indicator sets for both developing and developed countries may be unwise. Well-established, developed institutions may collect data with strong resource support and perform with greater continuity. On the other hand, developing countries face difficulties in focusing on what indicators are necessary for them and how these can be selected. There may be various reasons for such a situation. First, data is limited, while expansive data collection can be expensive. Second, awareness of the indicators and their purposes is low. Third, multiple sectors have the responsibility to gather different data, making coordination between sectoral stakeholders challenging. Therefore, the research question of this study asks which indicators are relevant for application in the context of metropolitan cities in developing countries, taking Greater Jakarta as a case study. Three sub-research questions guide the study: (a) what indicators are found in the literature that are differentiated in their scale of application?; (b) what are the criteria for selecting a category set of indicators for developing countries?; and (c) what categories of indicators are suitable for implementation in such countries?
Considering these questions, the study aims to explore context-specific indicators for developing countries. The paper is developed with the following steps: (i) synthesis and selection of various indicators from the literature; (ii) development of selection criteria; (iii) evaluation of the chosen indicators based on the criteria; and (iv) selection of the indicators which need to be applied in the context of developing countries using expert judgment. The following section will explain the research methodology, followed by presentation of the results. Section 4 analyses the results and discusses the important findings. Finally, the conclusions are drawn.
2. MATERIALS AND METHODS
To answer the research question, quantitative expert judgement methods were developed to rate the preselected set of categories. This section explains how the indicators and criteria were synthesised from the literature, how the data were collected from the respondents, and how they were analysed in order to obtain the results.
2.1. Literature Review to Select the Indicators and Criteria
The indicators selected were taken from relevant studies which discuss urban sustainable development indicators (urban SDIs) and Sustainable Transport Indicators (STIs). The snowball sampling technique was applied to identify the relevant papers. Two papers were used as starting point: Sdoukopoulos et al. [10] and Haghshenas and Vaziri [23]. The first paper was based 78 studies, which were evaluated in terms of the indicators used to measure sustainable transport development. With regard to the work of Haghshenas and Vaziri [23], we evaluated the composite index developed, which included 22 standardised sets of indicators. From these studies, we identified which papers contained detailed indicators and selected these for our sampling frame.
To make the selection, we first read the paper title and abstract. If these indicated that detailed indicators were included in the study, we investigated these further. We then compiled the indicators presented in the paper on our database, categorising them into urban SDIs and STIs. In total, 35 papers were identified covering the period 1999 to 2021 (Table 1). The location of the case studies varied, but they mostly concentrated on developed countries, such as Canada, Norway, France, and Ireland, with only a few focusing on developing countries.
| No | Author, Year | Location | Urban SDI | STI | Scale |
|---|---|---|---|---|---|
| 1 | Lautso and Toivanen, 1999 [15] | Helsinki | 0 | 28 | Regional |
| 2 | Nicolas et al., 2003 [19] | Lyon | 0 | 18 | Regional |
| 3 | Gilbert et al., 2003 [24] | Canada | 0 | 14 | National |
| 4 | Hezri and Hasan, 2004 [25] | Selangor | 30 | 2 | Regional |
| 5 | Rassafi and Vaziri, 2005 [26] | Developing countries | 0 | 18 | National |
| 6 | Zegras, 2006 [27] | Sao Paulo | 25 | 15 | Regional |
| 7 | Litman, 2007 [11] | Canada | 34 | 23 | National |
| 8 | Savelson et al., 2008 [28] | Halifax | 0 | 14 | Regional |
| 9 | Moles et al., 2008 [29] | Irish cities, Ireland | 40 | 10 | Regional |
| 10 | Litman, 2008 [12] | Canada | 30 | 23 | National |
| 11 | Appleton and Davies, 2008 [16] | Canadian cities | 0 | 17 | City |
| 12 | Doody et al., 2009 [30] | Irish city, Ireland | 36 | 3 | City |
| 13 | Li et al., 2009 [31] | Jining City | 52 | 1 | City |
| 14 | Silva et al., 2010 [17] | Sao Paulo | 37 | 21 | Regional |
| 15 | Castillo and Pitfield, 2010 [14] | England | 20 | 20 | Regional |
| 16 | Tanguay et al., 2010 [32] | Western countries | 29 | 1 | City |
| 17 | Mascarenhas et al., 2010 [33] | Portugal | 49 | 4 | Regional |
| 18 | Kane, 2010 [34] | Cape Town | 0 | 18 | City |
| 19 | Litman, 2011 [35] | Canada | 40 | 26 | National |
| 20 | Zito and Salvo, 2011 [36] | European cities | 0 | 32 | City |
| 21 | Haghshenas and Vaziri, 2012 [23] | Developing countries | 0 | 22 | City |
| 22 | Jeon et al., 2005, 2013 [37, 38] | Atlanta | 0 | 30 | Regional |
| 23 | Shen and Zhou, 2014 [9] | China | 105 | 3 | National |
| 24 | Santos and Ribeiro, 2015 [39] | Rio de Janeiro | 0 | 8 | Regional |
| 25 | Blumenfeld, 2015 [40] | Sao Paulo | 0 | 16 | City |
| 26 | Gudmundsson and Regmi, 2017 [18] | Asian cities | 0 | 2 | City |
| 27 | Jain and Tiwari, 2017 [13] | India cities | 0 | 35 | City |
| 28 | Munira and Santoso, 2017 [41] | Dhaka | 0 | 14 | City |
| 29 | Danielis et al., 2017 [42] | Italy | 0 | 16 | City |
| 30 | Lopez-Carreiro and Monzon, 2018 [43] | Spanish cities | 0 | 16 | City |
| 31 | Sdoukopoulos et al., 2019 [10] | Worldwide | 0 | 46 | City |
| 32 | Zope et al., 2019 [44] | Indian cities | 0 | 19 | City |
| 33 | Hipogrosso and Nesmachnow, 2020 [45] | Montevideo | 0 | 9 | City |
| 34 | Regmi, 2020 [22] | Asian cities | 0 | 10 | City |
| 35 | Rasca and Hogli Major, 2021 [46] |
Norwegian towns | 0 | 25 | City |
| Total | 988 | 527 |
As shown in Table 1, 988 indicators were collected from urban SDIs and 527 from STIs. Different scales of application for the indicators were found, with the papers referring to cities or regions, or both. Some papers were directly linked to applications in developing countries, such as those of Jain [13], Munira [41] and Zope [44], based on a variety of application scales. Comparing the level of application, the sets of indicators were applied on a multitude of spatial levels (i.e., local, regional, and national). Eleven initiatives were applied on a regional scale, 18 on a city scale, with the others on a national scale. Moreover, the papers reflect the utilisation of the indicators covered in two common documents, i.e., SDI and STI. A total of 35 papers from the last two decades were considered sufficient for analysis.
2.1.1. Indicator Selection
To select the indicators for the study, selected papers from the database were read to specify these. Indicators were then filtered out and recorded on the database. To focus on transport-related indicators, those considered irrelevant, such as percentage of educated people, number of hospitals, or tonnage of garbage, were omitted from the further analysis. The scales of the indicator implementation were also noted. Subsequently, we focused on the indicators applied in the context of transport (STIs). Indicators developed at the city scale and applicable in developing countries were preferred.
In the following step, the indicators were categorised into three dimensions of sustainable development: environmental, social, and economic). Indicators with close similarities were then grouped into one categorised indicator. For instance, PM (particulate) emissions, NO2 emissions, and number of days exceeding the air pollution limits for ozone were grouped in the same category, air pollution. The selected indicators were then rechecked with 35 studied articles. If the indicators were only accounted for in a few source articles, they were combined into other categories. In this step, a final total of 22 indicators was selected: eight environmental, ten social, and four economic)
2.1.2. Criteria Selection
Some researchers have categorised the criteria based on their level of strategic implementation [47], and on function-aspect [48]. Ledoux [47] classifies them in terms of sustainable development strategy, policy development implementation and analysis. On the other hand, Joumard [48] classifies them by function and by aspects such as representation (measurement), operation (monitoring), policy, and application (management).
To select which indicators were suitable for developing countries, six criteria were stipulated, based on the literature. Castillo and Pitfield [14] include four criteria for indicator selection, namely (a) measurability, (b) ease and speed of availability, (c) interpretability, and (d) transport impact-specific. Other authors suggest similar criteria: (a) relevance, (b) data availability and measurability, (c) validity, (d) sensitivity, (e) transparency, (f) independence, and (g) standardisation [11, 23]. The criterion of public participation was introduced by Doody et al. and Shen and Zhou to ensure local wisdom and a linkage between policymakers and those taking action [9, 30]. A description of the selected criteria is given in Table 2.
The selected criteria are also appropriate for evaluation/assessment purposes, since they are similar to the SMART (Specific-Measurable-Achievable-Relevant-Time Bound) criteria developed by Doran [49]. The interpretability criterion is similar to the specific one because it considers that the indicator should be easy to understand and not biased. The data availability criterion is related to measurability and is able to be quantified. Continuality is similar to achievability, while the goal-orientation criterion is similar to the time-bound one. The study will further define which criteria in Table 2 are more relevant than others.
These criteria were considered sufficiently comprehensive to cover common aspects from the planning function to that of implementation, as well as covering strategic-level decision making and the operational implementation level. For instance, the participation and goal-orientation criteria were categorised as being at the strategic level and more functional in representation. On the other hand, data availability, relevance and interpretability are more at the implementation level and operational function.
2.2. Data Collection Methods
A questionnaire was developed to elicit responses from the selected informants (respondents). The purposive sampling technique was used to identify the respondents included in the sampling frame, who were selected based on their expertise and knowledge, gained either from formal or informal education, related to transport development. Therefore, the respondents were assumed to fully comprehend the issues and indicators. They varied with regard to the organisation they worked for, their level of education, and years of professional experience. The number of respondents is 25, with the characteristics shown in Table 3.
| Criterion | Description |
|---|---|
| Interpretability (C1) | Indicators should be clear, unambiguous, easy to understand, specific, and measurable |
| Data Availability (C2) | Indicators should be reliable, cost-reasonable, and easy to calculate/predict |
| Continuality (C3) | Indicators should ensure continuality of evaluation with reasonable monitoring cost or availability of time series |
| Relevance (C4) | Indicators should be suitable for application in developing countries |
| Goal-Orientation (C5) | Indicators should have a clear scope and connect to the achievement of Sustainable Development Goals (SDG) |
| Participation (C6) | Indicators should promote public participation and be relevant to public perception |
| - | N (Total: 25) | % |
|---|---|---|
| Institution | ||
| Government | 12 | 48% |
| Private Sector | 5 | 20% |
| Academia | 8 | 32% |
| Education | ||
| Master’s | 12 | 48% |
| Professor/Doctorate | 13 | 52% |
| Years of experience | ||
| 0-10 | 8 | 32% |
| 10-20 | 13 | 52% |
| >20 | 4 | 16% |
The questionnaire consisted of two parts. The first measured the level of importance of the six criteria by using the pairwise comparison method. The second part evaluated whether the selected indicators were contextual for use based on the criteria. The description of the criteria and an explanation of how to compare them were given in the questionnaire. The respondents were allowed to discuss and give feedback to the author during the data collection.
Fifteen questions related to the six criteria were answered by pairwise comparison. Five scales/levels of response were developed on a Likert scale from 1 to 5, 3 being ‘moderately agree or disagree’, 5 ‘strongly agree’ and 1 ‘strongly disagree’. The responses to the questionnaire were then validated using the consistency ratio (CR). The pairwise comparison of Saaty [50] was utilised, since this can be used to effectively develop the weight of several criteria. The Likert scale was used due to its practicability. The respondents returned the questionnaire within the time range of a day to 2 weeks.
In total, 25 respondents completed and returned the questionnaire. The majority of them had an experience of working in a transport-related institution or conducting research on transport for more than 10 years. They came from government institutions or from non-governmental organizations, and international and national associations and universities. In terms of education, all the respondents held a master’s degree or doctorate.
2.3. Data Analysis
The data obtained from the questionnaire were analysed in two ways. First, descriptive analysis was employed to obtain information regarding the summary of the indicators, weighted criteria, and preferred indicators. This produced the mean average and weighted calculation of the indicators. For example, the final numbers to be used for scoring were compiled from the answers of the 25 respondents.
Second, a pairwise comparison judgement of each respondent was made, calculated separately to obtain the weight of each criterion for each respondent. Each respondent’s version of the sets of weights was then calculated from the mean average to determine the weight scores of the six criteria. A set of matrices was composed of the judgment values. The weight of each criterion was the eigenvector of the matrices [50, 51]. The consistency index (CI) was calculated to first determine the highest eigenvalue of the matrix λmax. The consistency ratio should be < 0.1 for the matrices to be accepted. In this study, all the respondents’ judgments had a CR below 0.09 (average CR of 0.08).
The final score of each indicator was calculated by the mean average of the response from each respondent and weighted by the determined weight scores of each criterion.
 |
(1) |
a, b, c, d, e, f = mean average of the Likert scale scores of Interpretability, Data Availability, Continuality, Relevance, Goal-Orientation, and Participation.
W1, W2, W3, W4, W5, W6 = weight scores of Interpretability, Data Availability, Continuality, Relevance, Goal- Orientation, and Participation.
3. RESULTS
This section presents the main results of the study used to answer the sub-research questions. First, the results regarding a set of indicators extracted from the literature are shown, followed by the elaboration of the criteria to select the indicators, and the selected context-based indicators from the interviews.
3.1. Comparison of the Indicators Found in the Literature
In total, 22 indicators were extracted from 35 papers. Three dimensions of sustainable development were used to categorise the indicators: (a) environmental, (b) social, and (c) economic. The environmental dimension contains categories that elaborate natural, ecological, and biodiversity aspects of urban transport development, such as pollution, energy, and nature sustainability [15, 31]. The economic dimension includes indicators evaluating urban transport’s economic and financial aspects, such as subsidies, costs and tariffs, and overall urban productivity [52]. These two dimensions are commonly quantifiable and can be easily measured.
Within the social dimension, indicators represent multiple facets of people’s quality of life and social welfare [ 52 ], including other social aspects of urban transport, such as safety, security, accessibility, and social equity. This dimension contains indicators that are more difficult to quantify and measure. The safety category includes all extracted indicators that evaluate transport system performance in preventing casualties and fatalities in traffic crashes [ 44 ]. The accessibility indicators refer to the easiness of people with different social and demographic backgrounds to access transport services and public facilities [ 28 , 38 ]. The mobility indicators evaluate the ease of moving from one place to another by reducing travel time or improving transport service availability [ 2 , 9 ]. The non-motorised modes category links to social inclusion indicators that promote walking and cycling to all citizens [ 17 , 40 ]. In addition, the public transport quality indicators evaluate people’s satisfaction with the transport service.
Eight indicators related to the environmental dimension, while the social and economic dimensions consisted of ten and four indicators, respectively. The indicators in the social dimension were more varied in their use than those in the economic dimension. For instance, social equity (S3) was only addressed in two papers, compared to safety (S1), which was addressed in 25 papers. The economic dimension indicator was less varied and could be grouped into four indicators.
Table 4 shows that Safety (S1) was the most frequent indicator, included in 25 papers. Next, the Transport Cost (E2) indicator was found in 24 papers, followed by 23 papers that contained the Air Pollution (L1) indicator and 20 that included both energy consumption and land consumption indicators. As the table shows, some indicators are less frequent in the literature; for example, Impact on Habitat/Wildlife (L7), and Impact on Cultural Heritage Sites (L8), with 5 and 6 papers, respectively.
The results show that based on the studies, 22 categories of indicators can be synthesised, which commonly cover all transport-related sustainability issues. The categorisation is practical as a framework for developing sets of indicators for sustainable transport [10, 32].
3.2. Criteria to Select Context-based Indicators
Table 5 shows respondents’ preferences for the criteria used to select the appropriate indicators. Relevance (C4) is the most weighted criteria, at 23%, followed by Data Availability (C2), Interpretability (C1), Goal-Orientation (C5), Continuality (C3), and Participation (C6). The difference in significance between the highest and lowest (23% vs. 11%) could be divided into a high importance category (18% to 23%), and low importance category (11% to 15% weight scores).
The Relevance (C4) criterion indicates that an indicator is suitable to be applied in developing countries based on their socio-economic situation, and that the context of cities in developing countries is fully considered in developing the indicators. Contextuality is also represented by Data Availability (C2). In the context of developing countries, the availability and quality of data is generally limited compared to western developed countries. Since the expert understands the data situation in Indonesia, it is suggested that data availability should be closely considered.
| - | Code | Categorised Indicator | Number of Articles using the Indicator | % of Articles using the Indicator (Out of 35 Studies) | Number of Operational Indicators |
|---|---|---|---|---|---|
| Environmental | L1 | Air Pollution | 23 | 66% | 54 |
| L2 | GHG Emissions | 15 | 43% | 21 | |
| L3 | Noise Level | 14 | 40% | 18 | |
| L4 | Energy Consumption | 20 | 57% | 33 | |
| L5 | Renewable Energy + Green Vehicle | 10 | 29% | 23 | |
| L6 | Land Consumption | 20 | 57% | 43 | |
| L7 | Impact on Wildlife Habitat | 5 | 14% | 8 | |
| L8 | Impact on Cultural Heritage Sites | 6 | 17% | 9 | |
| Social | S1 | Safety | 25 | 71% | 47 |
| S2 | Accessibility | 17 | 49% | 36 | |
| S3 | Social Equity | 2 | 6% | 6 | |
| S4 | Mobility | 19 | 54% | 40 | |
| S5 | Citizen Participation | 4 | 11% | 6 | |
| S6 | Security | 13 | 37% | 23 | |
| S7 | Non-motorised Modes | 7 | 20% | 10 | |
| S8 | Public Transport Quality | 8 | 23% | 10 | |
| S9 | Public Transport Mode Share | 4 | 11% | 5 | |
| S10 | Planning, Policy, and Institution | 10 | 29% | 21 | |
| Economic | E1 | Economic Productivity | 17 | 49% | 71 |
| E2 | Transport Cost (Passenger Benefit) | 24 | 69% | 42 | |
| E3 | Transport Efficiency (Provider Benefit) | 16 | 46% | 25 | |
| E4 | Public Expenditure, Investment, and Subsidies (Government) | 14 | 40% | 32 |
| Criterion | Weighted Score |
|---|---|
| Interpretability (C1) | 0.18 |
| Data Availability (C2) | 0.21 |
| Continuality (C3) | 0.13 |
| Relevance (C4) | 0.23 |
| Goal-Orientation (C5) | 0.14 |
| Participation (C6) | 0.11 |
In addition, Interpretability (C1) also scored highly. Respondents suggest it has high importance because of the complexity of the Jakarta urban transport system. Considering the multi-sectoral involvement and quality of human resources, it is necessary that the indicators be clear, unambiguous, easy to understand, specific, and measurable.
The findings also show that Participation (C6), Continuality (C3), and Goal-Orientation (C5) have low scores. In the context of cities in developing countries, the respondent does not consider public participation to be an effective consideration in selecting sustainable transport indicators. Public perception is also not considered to be a common criterion for establishing an indicator’s importance, neither the idea of continuous indicators to be essential in the context of Indonesia, since they require institutional and policy continuity.
3.3. Definition of Context-based Indicators for Metropolitan Areas in Developing Countries
High weighted criteria scores reflect that the indicator is more contextual for use in developing countries. High scores indicate that the indicators can be considered, have high importance, and have strong agreement with the six contextuality criteria. The results of the weighted average for each categorised indicator are shown on Table 6.
| Category Indicator | Interpretability | Data Availability | Continuality | Relevance | Goal-Orientation | Participation | Average | Score (weighted) | Rank | |
|---|---|---|---|---|---|---|---|---|---|---|
| (1) | (2) | (3) | (4) | (5) | (6) | (7) | (8) | (9) | (10) | (11) |
| L1 | Air Pollution | 4.48 | 4.12 | 3.65 | 4.52 | 4.23 | 2.65 | 3.94 | 4.06 | 5 |
| L2 | GHG Emissions | 3.88 | 3.28 | 3.39 | 4.04 | 4.05 | 2.74 | 3.56 | 3.62 | 14 |
| L3 | Noise Level | 3.56 | 3.36 | 3.13 | 4.17 | 3.77 | 2.96 | 3.49 | 3.56 | 16 |
| L4 | Energy Consumption | 4.20 | 4.12 | 3.74 | 4.22 | 4.09 | 3.13 | 3.92 | 3.99 | 7 |
| L5 | Renewable Energy + Green Vehicles | 3.40 | 3.08 | 3.43 | 3.74 | 3.82 | 2.78 | 3.38 | 3.41 | 19 |
| L6 | Land Consumption | 3.52 | 3.84 | 3.83 | 3.39 | 3.09 | 2.70 | 3.39 | 3.44 | 18 |
| L7 | Impact on Wildlife Habitat | 2.91 | 2.70 | 3.00 | 3.00 | 3.05 | 2.48 | 2.86 | 2.87 | 22 |
| L8 | Impact on Cultural Heritage Sites | 3.09 | 2.96 | 2.78 | 2.87 | 2.95 | 2.43 | 2.85 | 2.88 | 21 |
| S1 | Safety | 4.30 | 4.17 | 4.35 | 4.39 | 4.23 | 3.57 | 4.17 | 4.21 | 1 |
| S2 | Accessibility | 4.04 | 3.96 | 3.87 | 4.35 | 4.14 | 3.65 | 4.00 | 4.04 | 6 |
| S3 | Social Equity | 3.43 | 3.13 | 3.35 | 3.96 | 3.95 | 3.83 | 3.61 | 3.60 | 15 |
| S4 | Mobility | 4.13 | 3.57 | 3.74 | 4.39 | 4.14 | 3.83 | 3.96 | 3.99 | 8 |
| S5 | Citizen Participation | 3.13 | 2.78 | 3.00 | 3.52 | 3.45 | 4.09 | 3.33 | 3.29 | 20 |
| S6 | Security | 3.43 | 3.17 | 3.22 | 3.87 | 3.59 | 3.65 | 3.49 | 3.50 | 17 |
| S7 | Non-motorised Modes | 4.04 | 3.61 | 3.52 | 4.00 | 3.91 | 3.52 | 3.77 | 3.80 | 11 |
| S8 | Public Transport Quality | 4.22 | 4.09 | 4.04 | 4.39 | 4.18 | 3.78 | 4.12 | 4.15 | 2 |
| S9 | Public Transport Mode Share | 4.22 | 4.17 | 3.96 | 4.39 | 4.23 | 3.30 | 4.05 | 4.11 | 3 |
| S10 | Planning, Policy, and Institutions | 3.58 | 3.52 | 3.83 | 4.22 | 3.68 | 2.70 | 3.59 | 3.66 | 12 |
| E1 | Economic Productivity | 3.52 | 3.74 | 3.83 | 4.00 | 3.55 | 2.91 | 3.59 | 3.65 | 13 |
| E2 | Transport Cost (Passenger Benefits) | 4.26 | 3.74 | 4.04 | 4.30 | 4.09 | 3.96 | 4.07 | 4.08 | 4 |
| E3 | Transport Efficiency (Provider Benefits) | 4.09 | 4.04 | 3.83 | 4.00 | 3.86 | 2.87 | 3.78 | 3.85 | 10 |
| E4 | Public Expenditure, Investment, and Subsidies (government) | 4.00 | 4.39 | 3.91 | 4.00 | 3.68 | 2.65 | 3.77 | 3.87 | 9 |
The final aggregation of each score based on the criteria are shown in columns 3 to 8. Columns 9 and 10 indicate that the scores calculated by the mean average of the six criteria are not significantly different from those calculated by the weighted criteria. The weighted scores (calculated by Equation 1) are applied for further discussion.
Six indicators have scores higher than 4 (out of the 5 scales): Safety (S1), Public Transport Quality (S8), Public Transport Mode Share (S9), Transport Cost (E2), Air Pollution (L1), and Accessibility (S2). These could be considered to be contextual indicators. The respondents strongly agreed that these indicators were very important, based on the criteria. The findings show that Safety (S1) had the highest score of 4.21, with score over 4 also for Interpretability (4.30), Data Availability (4.17), Continuality (4.35), and Goal-Orientation (4.23). Most of the high-scoring indicators were in the social dimension.
Four out of the ten indicators in the social dimension scored more than 4, and no indicator had a score below 3. Moreover, eight of the environmental indicators are quite varied. One indicator had a score over 4 (L1), five indicators score over 3, and two with scores below 3.
In terms of low scores, the Impact on Wildlife Habitat (L7) and Impact on Cultural Heritage Site (L8) indicator scores were below 3, at 2.87 and 2.88, respectively. Both are environmental dimension indicators. It seems that the respondents did not consider these indicators to be important in the context of cities in developing countries.
4. DISCUSSION
This section discusses the main findings of the previous section. It includes why specific indicators are prominent in the literature, which criteria have mostly guided the indicator selection, and how context-based indicators have been developed and implemented.
4.1. Variability of Indicators
Sustainable transport indicators (STIs) are closely related to Sustainable Development Indicators (SDIs), since STIs could also be used for evaluating the sustainability of cities [10, 23]. The 22 categorised indicators are considered more practical and could cover all transport-related issues of sustainability, being similar to the 47 themes found by Sdoukopoulos et al. [10]. Most indicators used by global initiatives are Safety (S1), Air Pollution (L1), Energy Consumption (L4), Land Consumption (L6), and Mobility (S4). This is also similar to the finding of [10], that Safety, Mobility, GHG Emissions, Accessibility, and Fossil Fuel Energy consumption were the five indicators used in global initiatives.
Impact on Wildlife Habitat (L7) and Impact on Cultural Heritage Sites (L8) are used infrequently as they measure very specific aspects and are bound to very specific locations. For that reason, Litman (2007) explains that these indicators are specialised and used for specific needs and objectives. For example, a transport project that crosses conservation forest areas will disrupt the wildlife habitat and cultural heritage sites. These indicators are more suitable in the context of rural rather than urban areas.
Indicators such as Social Equity, Citizen Participation and Public Transport Mode Share address similar issues to the Accessibility (S2) indicator. Public Transport Mode Share (S9) and Public Transport Quality also address the same issues. Therefore, social dimension indicators are mostly more varied in their level of operational use by global initiatives. Sdoukopoulos and Pitsiava-Latinopoulou [53] highlighted factors resulting in the variability of indicators to be the complexity of the transport system, the characteristics of an area, and the availability of data.
4.1.1. Criteria
From our findings, the criteria were weighted to calculate the scores and establish which indicators were contextual for application in metropolitan areas in developing countries. Relevance (C4) was the highest scoring criterion as it reflects the aspect of local specialism. For example, urban areas differ from rural ones in the way that the inhabitants’ socio-economic status could result in a particular aspect being valued more than others; for example, air pollution is of more concern to urban inhabitants. This criterion was used by [23] to ensure that the indicators showed one aspect of sustainable transport. Relevance is important, as indicated by Litman [11], who suggests that indicators should focus on and be consistent with the issues addressed. Indicators also should have strong relevance to the policies and goals of sustainability [24, 54].
The next highest score criterion from the findings was Data Availability (C2). In developing countries, this is different from western developed countries. The criterion suggests that indicators should be easy to collect, easy to measure and predict, and should be available at a reasonable cost and within a reasonable time. Joumard et al. [48] categorise data availability and measurability into operation (monitoring) related criteria. The initiative by ELASTIC [14] uses three different criteria, measurability, ease of availability, and speed of availability, to ensure that the indicators are operational. Without good data availability, indicators are less operational and cannot be used for evaluation/assessment purposes [12, 24].
The result shows that the low-scoring criteria are Participation, Continuality, and Goal-Orientation. The Participation (C6) criterion was introduced by Doody et al. [30] to ensure that the indicators were inclusive and followed a bottom-up process from the community to policymakers. The idea that indicators should come from a democratic, inclusive process is less popular in Indonesia, where most of the public policy is decided by the government. The respondents also considered that sustainable development was exclusively a government obligation.
In addition, Continuality (C3), which is defined as an indicator that should have a continuous evaluation at reasonable monitoring costs, is also non-comprehensive and, to some extent, covered by the Data Availability (C2) criterion. The low scores might reflect that some respondents assumed that the criterion was unnecessary, since it was already covered by others.
4.1.2. High Scoring Indicators
Based on Table 6, high scoring indicators of over 4 were Safety (S1), Public Transport Quality (S8), Public Transport Mode Share (S9), Transport Cost (E2), Air Pollution (L1), and Accessibility (S2). Most of these are related to the social dimension, but other dimensions are also covered.
Table 7 gives examples of the operational indicators used in various initiatives. Although such indicators have many variations and types, they have the same objective of measuring similar sustainability factors. For example, safety indicators could be measured by different operational indicators, such as the number of road fatalities per 100,000 inhabitants or the number of road accidents, among others.
Sustainable transport systems should have the highest practical standards of safety and security. Safety indicators are commonly used by global initiatives (71%) (Table 4). Richardson [55] indicates that safety is the main concern from both economic and societal perspectives, and it has a high importance score. Transport safety is affected by several factors, such as human, vehicle, and road and environmental ones [56]. Safety, referring to being safe on the road and transport mode, avoiding accidents and being protected against crime, is also an effective indicator to improve the quality of life [57].
Concern with public transport is above average, as indicated by the high scores of both the Public Transport Quality (S8) and Mode Share (S9) indicators. Sustainable transport systems promote the use of public transport services. In urban areas, where the majority of transport users are commuters, public transport plays a significant role in urban sustainability. Many cities with land-use policies that clearly separate work and residential zones will have to deal with commuter transport. The coverage of the public transport system, as well as its quality, as discussed by Kane [34], are important indicators, as they can ensure the effectiveness of citizen mobility. Miller [58] explains how public transport can effectively contribute to city sustainability. First, it can provide more energy efficiency in urban settings compared to private car travel. Second, it reduces gas emissions and pollution, which affect the environment. It also significantly reduces the operating cost per unit of travel compared to private cars, and finally, it promotes inclusivity, which is good for citizens’ social experience.
Since the nature of public transport is a service, indicators developed are usually in a qualitative sense [41]. The air pollution indicator is commonly used to quantify the impact of the transport system on the environment. This quantitative indicator is used by the majority of global initiatives (66%) (Table 4). The indicator is directly related to sustainability, since it measures its environmental dimension. Litman [11] rates air and noise exposure to health quality as the most important indicators that should usually be employed.
| Contextual Indicator | Examples of Operational Indicators | |
|---|---|---|
| L1 | Air Pollution | • PM (particulate) emissions • NO2 emissions • Number of days exceeding the air pollution limits for ozone |
| S1 | Safety | • Number of road fatalities per 100,000 inhabitants. • Number of road accidents |
| S2 | Accessibility | • Share of population living within 300–500 m of public transport stations/stops • Access to urban functions (jobs, schools, retail, health) |
| S8 | Public Transport Quality | • Public transport reliability • Waiting times and headways |
| S9 | Mode Share | • Share of trips by public transport • Mode share • Vehicle growth |
| E2 | Transport Cost (passenger benefit) | • Fuel prices and taxes • Public transport tariffs • Annual average expenditure on transport |
| S.No | SUTI Indicator | Related Indicators in this Study | Score |
|---|---|---|---|
| 1 | The extent to which transport plans cover public transport, intermodal facilities, and infrastructure for active modes | Accessibility (S2) | 4.04 |
| 2 | Modal share of active and public transport in commuting | Public Transport Mode Share (S9) | 4.11 |
| 3 | Convenient access to public transport service | Accessibility (S2) | 4.04 |
| 4 | Public transport quality and reliability | Public Transport Quality (S8) | 4,15 |
| 5 | Traffic fatalities per 100,000 inhabitants | Safety (S1) | 4.21 |
| 6 | Affordability – travel costs as a proportion of income | Transport Cost (E2) | 4.08 |
| 7 | Operational costs of the public transport system | Transport Efficiency (E3) | 3.85 |
| 8 | Investment in public transport systems | Government Expenditure, Investment (E4) | 3.87 |
| 9 | Air quality in cities (PM 10) | Air Pollution (L1) | 4.06 |
| 10 | Greenhouse gas emissions from transport | GHG Emissions (L2) | 3.62 |
Transport cost is also a quantitative indicator commonly used to measure the economic dimensions of transport sustainability. Affordable transport costs are considered to be more sustainable than expensive transport. Most initiatives comprise economic indicators, be they macroeconomic indicators, passenger transport costs, service provider transport efficiency, government investment expenditure, or subsidies. Transport-related economic indicators are mainly categorised by four different stakeholder perspectives: people, passengers, service providers, and the government [23, 35]. The results of our study show that the passenger view is considered more important than other perspectives. Therefore, in cities in developing countries, the perspective of transport users should not be neglected.
Comparing the SUTI indicators, this study finds similarities with SUTI initiatives, in which the indicators achieving high scores (see Table 8) are Accessibility (S2), Public Transport Mode Share (S9), Public Transport Quality (S8), Safety (S1), Transport Cost (E2) and Air Pollution (L1). All the highly important indicators are already covered by SUTI. However, SUTI does not yet cover the Energy Consumption (L4) indicator, which is regarded as an important indicator in this study. Energy Consumption (L4) scores higher than Transport Efficiency by Provider (E3), Government Investment (E4), and Greenhouse Gas Emissions (L2). The respondents believed that transport sustainability was more effectively measured by energy consumption indicators, in line with SDG 7, Affordable and Clean Energy and SDG 12, Responsible Consumption and Production. Therefore, this paper suggests that SUTI covers energy consumption factors in its protocol.
Our findings also show that the two environmental indicators scored low (below 3). Transport impact on wildlife habitat and impact on cultural heritage sites does not seem to be considered important by the respondents. This is alarming since the environmental dimension should be given high consideration in the context of urban transport sustainability. The first reason, the majority of land use in the urban sphere comprises the built environment. Jakarta metropolitan area has 39% of built land use, while DKI Jakarta Province, like the central business district, comprises more than 90% built environment. Therefore, the respondents considered L7 and L8 less important due to being less appropriate for the urban context. Second, the two indicators are also less popular among global initiatives (for only six out of the 35 authors; Table 4). The third reason is that neither indicator is directly related to human interests, although classified as an environmental dimension, and their impact on human welfare is not as direct as air and noise pollution or energy and land consumption [9, 31]. Another reason is that the indicators are too narrow and specialised for special needs or objectives [11].
CONCLUSION
This study aimed to explore a category set of indicators for metropolitan cities in developing countries, taking Jakarta as a case study. It shows that some categories are relevant for measuring sustainable transport development in such a context, including air pollution, safety, accessibility, public transport quality, mode share, and transport cost. However, some indicator categories may be less contextual than others. This means that some categories may be unable to specify what sustainability elements to measure, or some may be less operational because their suitability for implementation is limited by the context.
Developing countries such as Indonesia lack the resources to apply a comprehensive set of indicators for transport sustainability [7]. This research suggests six criteria to identify which indicator categories could be included in the set. The criterion set is useful for selecting indicators that focus on what strategic issues matter, and how the context will be operationalised effectively. The strategic aspect deals with the future challenges and vision of metropolitan areas. The operational aspect concentrates on resource limitations and institutional barriers to implementation, such as the time and money needed to collect the data.
As for research limitations, the study relied on an unsystematic review to extract indicators from the literature. As a result, some important papers may have been unintentionally ignored in the analysis because of the less than robust search process. A systematic literature review could help to extract the remaining relevant indicators. Moreover, the study used criteria that reflect both the strategic and implementation levels of the indicators identified. However, the research tends to concentrate on the implementation. A balanced adoption of criteria based on these levels is needed for the selection.
In future research, the indicator categories resulting from this study need to be further developed in actual application. The development process will require the inclusion of multiple stakeholders to specify the strategic use and to handle the constraints in implementation collectively. Moreover, future studies could focus on the comparison and integration of the suggested indicators with other sets of indicators or indexes (e.g., SUTI) to contribute to a context-specific based indicator for metropolitan areas in developing countries.
LIST OF ABBREVIATIONS
| SUTI | = Sustainable Urban Transport Index |
| STIs | = Sustainable Transport Indicators |
CONSENT FOR PUBLICATION
Not applicable
AVAILABILITY OF DATA AND MATERIAL
The authors confirm that the data supporting the findings of this study are available within the article.
FUNDING
None.
CONFLICT OF INTEREST
The authors declare that they have no competing interests.
ACKNOWLEDGEMENTS
The authors would like to thank Professor Wimpy Santoso and Dr. Raldi Koestoer for commenting on the conceptual design of the manuscript. They also wish to thank the officials from government institutions and various organisations. Academics from Institut Teknologi Bandung and the University of Indonesia also contributed by providing respondent information. In the School of Environment, University of Indonesia, we are very grateful to Dr. Budhi Soesilo for their administrative support.
