The Analysis of Dynamic O / D Adjustment for Bicycle Traffic Demand Estimation with AIMSUN Simulation Model : A Case Study of Nakhon Sawan Municipality in Thailand

Methods: The primary and secondary traffic data were collected to develop a virtual traffic simulation model with the use of the AIMSUN simulation software. The model validation method was carried out to adjust the origin and destination survey data (O/D matrix) by running dynamic O/D adjustment. The 99 replication scenarios were statistically examined and assessed using the goodness-of-fit test. The 9 measures, which were examined, included: 1) Root Mean Square Error (RMSE), 2) Root Mean Square Percentage Error (RMSPE%), 3) Mean Absolute Deviation (MAD), 4) Mean Bias Error (MBE), 5) Mean Percentage Error (MPE%), 6) Mean Absolute Percentage Error (MAPE%), 7) Coefficient of Determination (R2), 8) GEH Statistic (GEH), and 9) Thiel’s U Statistic (Theil’s U).


INTRODUCTION
The Thai government has placed importance on formulating policies to solve traffic congestion in a sustainable way.INRIX [1] assessed traffic conditions in 195 countries and revealed that in 2017 Thai drivers spent an average of 56 hours in traffic congestion during peak hours, compared to 61 hours in 2016.Although its traffic congestion was getting less over the two years, Thailand was still ranked the most traffic-congested country, out of 38 countries with the worst traffic in the world.Thus, the government has tried to solve the problem by launching an emergency plan to promote changes in the mode of transportation in every province of Thailand.The people that need to commute to areas within a 20 kilometer radius from their house are encouraged to use bicycles instead of private cars.However, there are still problems in the current bike route design because most bike routes were mainly designed based on the preferences of designers and existing cyclists, which sometimes might not cover the traffic demand of people in each community.This made private car drivers unwilling to change their transportation mode or use bicycles for daily transportation.After reviewing the results of bike route evaluation in many areas, it was found that the number of bike route users was lower than expected, the budget was not invested in worthwhile projects, and the design failures often caused traffic accidents [2].
Nakhon Sawan Municipality is one of the first cities in Thailand that seriously promote the use of bicycles.It is located in the central region, where the Ping and Nan Rivers converge and form the Chao Phraya River, the most important waterway of Thailand, as shown in Fig. (1).Its population density is 3,010 people per square kilometer [3].Nakhon Sawan Municipality is considered one of the largest cities in Thailand.Nakhon Sawan Municipality built the first bike route around Sawan Park for exercise purposes in 2013 [4].Then there was an initiative to extend and develop that bike route into the community's daily transportation network.The qualitative research was conducted using focus group discussions.The four groups of informants, consisting of 1) government officials, 2) designers, 3) users, and 4) press, were selected to share their opinions about bike routes.The results suggested that a circular bike route should be designed along the main roads and a shared bikeway should be built separately from the main traffic lane [2].However, the results of that research only reflected the traffic demands of 4 groups of people.The daily traffic demand of other groups of people had not been examined.Therefore, the present research aimed to analyze the estimated traffic demand in the road network of Nakhon Sawan Municipality with the use of the AIMSUN simulation program.The model validation method was also carried out to adjust the origin and destination survey data according to the traffic engineering principles.This is to ensure that the resulting data can be used to effectively design the bike route network that truly meets the community's daily traffic demand.This paper is organized as follows.Section 2 introduces the statement of problem.Section 3 presents the review of the literature.Section 4 describes how to survey data and generate an AIMSUN simulation model as well as explains the implementation of data analysis and model validation processes.Section 5 shows the results of running 99 replication scenarios of dynamic O/D adjustment, the results of statistical analysis, the acceptable deviation criteria, and the application of the multi-factor scoring method to the selection and ranking of O/D matrix adjustment.Sections 6 gives summary and conclusion of the present study.

STATEMENT OF PROBLEM
As the main transportation mode in Nakhon Sawan Municipality is private cars, it is important to analyze the traffic demand of private car users between origin and destination sub-areas and use the obtained results to design a bike route that meets the daily transportation needs.This will help motivate local people to use bicycles as their main transportation mode instead of private cars.According to the traffic engineering principles, the origin and destination survey can be carried out in many different ways through the use of questionnaire technique [5], license plate matching technique [6 -9], or Bluetooth MAC address readers [10].The present research used the license plate matching technique to conduct the origin and destination survey because it is the most common method that requires less budget and can be easily applied to other areas in Thailand.However, the results of the license plate matching in each checkpoint indicated that there are only a few matching entries, compared to the traffic volume during the survey period.The road network of Nakhon Sawan Municipality and other cities in Thailand is not systematically organized according to the road functional classification and hierarchy of movement [11], causing each sub-area to have multiple access routes.Thus, conducting a survey that covers all traffic routes is difficult and requires a large budget.

LITERATURE REVIEW
Traffic simulation is considered an important tool in traffic engineering.It is used to analyze and test the design or concept of research, which helps reduce cost and impact that may occur when compared to field test.Most research studies on traffic simulation model are associated with analyzing traffic behaviors and evaluating the design or improvement of the physical characteristics of roads.
Tran Vu Tu and Kazushi Sano [12] conducted an analysis of the impacts of scramble crossings in comparison with traditional crossings on the intersection performance by using the Paramics simulation model.John Sangster et al. [13] examined the operational aspects of the through-about intersection design and carried out the comparative analysis of overall intersection delays using the SYNCHRO traffic software.Lenorzer A. et al. [14] used the Aimsun micro simulator to evaluate new road design solutions through developing a new mixed-flow model, which aimed at providing a detailed and robust behavioral data and simulating sensitivity analysis that focused on the capacity and jam density as well as the application to busy junctions in a city [14].David Stanek [15] also studied the increased use of cycle tracks that leads to investigations of how to accommodate them at the intersection by using the VISSIM traffic analysis software.In addition, there are research studies on bicycle traffic simulation that are involved with the management of bike sharing systems.For example, Thibaut Dubernet and Kay W. Axhausen [16] developed the multi-agent activitybased simulation model to estimate the reactions of the demand to changes in the quality of the bike redistribution and relocation strategies.Robert M. Saltzman and Richard M. Bradford [17] investigated an appropriate number of docks and bikes to reduce operators' time and resources (including fossil fuels) in moving bikes from stations with an excess of bikes to those with shortages by using an animated discrete-event simulation model.
There are some studies that propose the procedures of model construction, calibration, and validation in order to ensure the accuracy and credibility of each virtual traffic model.For example, Oketch T. and Carrick M [18].suggested that the calibration effort involved comparing the model results to the field data that included traffic volume, movement, average travel time, and approach queues.Paramics uses a dynamic assignment procedure in which movements of vehicles through the network are governed by origin-destination matrices on the basis of various assignment techniques.The modeling exercise involved estimation of suitable origin-destination matrices which could replicate the observed traffic volumes and turning movement counts at selected intersections to acceptable levels.In addition, Chen-Ju Wu et al. [19] developed A procedure for constructing and calibrating a microsimulation model of a congested freeway with multiple vehicle classes by using AMSUN microsimulation model.They also suggested that the model construction and calibration procedure is composed of the following steps: 1) building the road geometry in AIMSUN, 2) collection of the traffic condition data from the PeMS database, 3) imputation of missing data, 4) estimation of onramp demand and off-ramp turning, 5) identification of recurring bottlenecks, 6) setting model parameters, and 7) adjustment of model parameters such that the simulation results match with the field measurements.
As the accuracy of the model is very important, the process of model validation is carried out in order to examine, assess, compare, and explore the relationships between observed traffic volume and modelled count data.According to the general simulation literature, a simulation model can be statistically validated using a goodness-of-fit test.UK Highways Agency [20], AIMSUN [21] and Xiao-Yun Lu et al. [22] indicated that two alternative analytic methods that are frequently applied to validation comparisons are: 1) the GEH statistic, which is a form of the Chi-squared statistic that incorporates both relative and absolute errors, and 2) the R Squared (R 2 ), which gives some measure of the goodness of model fit and the slope of the best fit regression line.Theil H [23]. proposed the Theil's U statistic measure that forecasts accuracy by comparing two auto-correlated time series with predicted values and observed values [23].Tomer Toledo and Haris N. Koutsopoulod [24] stated that Among a number of goodness of fit measures, the Root Mean Square Error (RMSE), the Root Mean Square Percentage Error (RMSPE%), the Mean Error (ME), the Mean Percentage Error (MPE%), and the Thiel's U measure (Theil's U) are the popular ones.Daiheng Ni et al. [25] also proposed Additional goodness of fit measures, which are the Mean Absolute Error (MAE) and the Mean Squared Error (MSE).Jonathan Annan et al. [26] suggested that the Mean Absolute Deviation (MAD) is an effective measure of the average of the absolute difference between the actual observations and the predicted variable in the time series.
However, the process of model validation can be conducted using different goodness-of-fit test measures.In each research study, only a few measures are used to analyze and describe the validity of traffic simulation model.Based on this knowledge gap, the present research aimed to study the application of AIMSUN simulation model in estimating traffic demand.Nine goodness-of-fit test measures were selected to use in the model validation process.A procedure of model calibration and validation relating to statistical measurement was developed.Then the scenario with the highest score, resulting from running 99 replication scenarios of dynamic O/D, was selected using the multi-factor scoring method.The multi-factor scoring method is a collection of quantitative methodologies that can be used to make a choice from a set of alternatives by using a set of two or more factors as decision choice criteria [27].This method has been widely used to prioritize and gives support to the management of project portfolios [28].It is also used to evaluate and select early stage technology and innovation the projects during the decision-making process [29].

Research Procedures
The process of traffic demand estimation was comprised of the following 7 steps: 1) collecting data

Field Data Collection
The traffic field survey was carried out to collect the data about functional classification and geometric design of road network, traffic volume, travel speed, travel time, delay time, traffic management, and origin-destination trips [11,30,31].The data about traffic behavior related to the campaign that encouraged people to change their daily transport mode from private car to bicycle were collected on weekends during 6:00 AM -1:00 PM [2].In terms of traffic volume, the mid-block counts were conducted in 15-minute intervals at 54 locations and the vehicles were categorized into 8 groups.The observed traffic volume was subsequently converted into Passenger Car Unit (PCU) before importing to database in form of Real Data Sets (ISO Format YYYY-MM-DDTHH:MM:SS).The origin-destination survey was carried out in 18 locations with the license plate matching techniques [6 -9] in order to examine the traffic demand in Nakhon Sawan Municipality that were classified into 23 sub-areas (17 internal sub-areas and 6 external sub-areas) according to land use purposes [30].The survey results are shown in Fig. (3).The data about travel speed, travel time, and delay time were collected using the test vehicle techniques [32].The Global Positioning System (GPS) and GPS car video recorder were installed to record traffic movements and display all related information while driving through the roads [33].

AIMSUN Simulation Model
AIMSUN is a traffic simulator that follows a microscopic simulation approach.The behavior of each vehicle in the network is continuously modelled throughout the simulation time period while it travelled through the traffic network, according to several vehicle behavior models such as car following and lane changing.The input data required by AIMSUN dynamic simulator is a simulation scenario.The simulation parameters are fixed values that describe the experiment and some variable parameters used to calibrate the models [21].In order to improve the accuracy of model, the O/D matrix data should be adjusted to observe traffic volume (Real Data Sets).The O/D matrix adjustment can be carried out by combining direct and indirect model estimators with other aggregated information related to O/D demand flows [34,35].The O/D matrix adjustment is based on a bi-level model solved heuristically by a gradient algorithm.It is a procedure for estimating an O/D matrix, from an a priori O/D matrix, using link traffic counts for which observed traffic volume (Real Data Sets) is available [36].

Construction of the AIMSUN Simulation Model
A 1:4,000 scale aerial photograph was imported into the ArcGIS program to create GIS road network map.Then the road network was digitized and the physical characteristics of the roads were recorded and linked to attribute data such as road (Link), junction (Node), number of lane, and lane width [37].After that, all the data were imported to create AIMSUN simulation model of the road network.Lastly, other traffic data, both primary and secondary, were separately inputted in each segment of road such as hierarchy of road, capacity, traffic variables, flows speeds, traffic management, driver behavior, and reaction time.The details are shown in Fig. (4).

Data Analysis and Validation Model
Considering the traffic demand adjustment, the results of the origin and destination (O/D) survey in form of O/D matrix were adjusted by running the dynamic O/D adjustment.The maximum number of iterations was set to 99 scenarios.The adjustment result of each scenario was clearly displayed in the O/D matrix adjustment process.Then the model validation process was carried out.All 99 scenarios were examined and evaluated using quantitative methods, which helped quantify the difference between the observed traffic volume (Real Data Sets) and the modelled count data.Quantitative validation can be performed with the use of statistical measurement.The general simulation literature includes a large number of approaches for the statistical validation of simulation models.The goodness-of-fit test was selected to use in the present research.The 9 measures that were used to quantify the model predictive accuracy consisted of 1) Root Mean Square Error (RMSE), 2) Root Mean Square Percentage Error (RMSPE%), 3) Mean Absolute Deviation (MAD), 4) Mean Bias Error (MBE), 5) Mean Percentage Error (MPE%), 6) Mean Absolute Percentage Error (MAPE%), 7) Coefficient of Determination (R 2 ), 8) GEH Statistic (GEH), and 9) Thiel's U Statistic (Theil's U).Suppose there are two processes Y i (observed traffic volume) and X i (modelled count data): Y 1 , Y 2 , …, Y n and X 1 , X 2 , …, X n , where n is the sample size.The description of each factor is listed in Table 1.The results were analyzed using the multi-factor scoring method.The O/D matrix adjustment output of the scenario with the highest score was set as the estimated traffic demand between sub-areas in Nakhon Sawan Municipality.
0 implies worst possible fit, and R 2 = 1 implies perfect fit.

Estimation of Traffic Demand by Using Dynamic O/D Adjustment
In order to simulate traffic situations, the origin-destination survey data during peak hours were imported into Demand Data in form of O/D matrix.The types of vehicles were also converted into Passenger Car Unit (PCU).The normal distribution was given for global arrivals.Then the traffic demand estimation was carried out by running dynamic O/D adjustment replication.The maximum number of iterations was set to 99 scenarios in order to find the O/D matrix adjustment output.Dynamic O/D adjustment is a procedure for adjusting a priori O/D matrix using traffic counts.It is used to adjust O/D matrix derived from demand predictions to agree with traffic volume observed (Real Data Sets, ISO Format YYYY-MM-DDTHH: MM:SS).The results and comparison of the O/D matrix adjustment of 99 scenarios are shown in Table 2 and Fig The  Fig. (5) shows that Scenario 1 and Scenario 2 have a similar graph pattern in each road segment (Object).The road segments with high traffic volume from modelled count data are obviously reflected through the graph pattern.In other words, the modelled count data of Scenario 1 is higher than the Real Data Sets.It reaches optimum point before sharply decreasing in Scenario 2. From Scenario 3 to Scenario 99, the graph slightly fluctuates with no clear pattern.This is in line with Fig. (6) that presents the relations between observed traffic volume (Real Data Sets) and modelled count data and reveals that the trend line of Scenario 1 tends to be distantly separate from that of other scenarios.
According to the analysis of mean absolute percentage error, it was found that the top 5 scenarios with the highest percentage error were 1) Scenario 1 (28.98%), 2) Scenario 2 (19.88%), 3) Scenario 3 (7.49%),4) Scenario 84 (17.41%), and 5) Scenario 87 (16.92%).The results indicated that the maximum number of iterations should be set at 2 (Scenario 2) or higher so as to remove significantly fluctuating data.The most accurate scenario could not be identified because the graph still slightly fluctuated and was unlikely to converge to the best value.Then, the model was further measured and statistically validated using the goodness-of-fit test.

Quantitative Techniques for Statistical Validation Model
The observed traffic volume (Real Data Sets) and the traffic volume from modelled count data of 99 scenarios were substituted in the formula of each 9 measures listed in Table 1 in order to quantify the model predictive accuracy.From the Box-and-Whisker Plot in Fig. (7), it was found that MPE(%) was the factor with the highest distribution value, followed by MAPE(%) and RMSE.MAD was found to have the highest outliers, as the abnormal values of some scenarios were distant from others in terms of upper extreme and lower extreme positions, which indicated a random sample from a population.However, running dynamic O/D adjustment with a maximum number of iteration of 4 (Scenario 4) could eliminate the outliers of all factor except for MAD.This was because the outliers of MAD were randomly generated, making it hard to clearly summarize the results.

Statistical Measurement Criteria
The statistical measurement values of the 99 scenarios were compared to the acceptable indicators of each factor in order to determine the statistical measurement criteria.The measurement values that were closest to the acceptable indicators of each factor were set as the upper limit values.Then these upper limit values were used to identify the lower limit values and acceptable ranges.The results suggested that, among all 99 scenarios, there were only 8 scenarios whose measurement values were in the acceptable ranges, as the details shown in Table 3.It was found that the measurement values of Scenario 23 reached the upper limit value of 4 factors: RMSE, MAD, R 2 , and Theil's U, and the lower limit value of RMSPE(%).On the other hand, the measurement values of Scenario 25 were found to reach the upper limit value of only one factor, MBE, and reached the lower limit value of 3 factors: MAPE(%), R 2 , and Average GEH.Moreover, the measurement values of Scenario 43, 47 and 48 were in the acceptable ranges, without reaching any upper or lower limit value.This made the selection of the most accurate scenario for each factor uncertain and difficult to decide.Thus, the multi-factor scoring method was additionally used in the decision-making process.

O/D Matrix Adjustment Resulting from Applying Multi-Factor Scoring Method
The multi-factor scoring process was conducted by determining the factor score of each 9 measures according to the statistical measurement value and converting it into the conversion value as shown in Equation 10 and Equation 11 in order to avoid bias in calculation and different error ranges.

(10) (11)
Where: ε i = absolute error i, Ai k = acceptable indicators of factor k, Vm i = measurement value of factor i, and Con i = conversion value of factor i.
Scoring weight was set based on an assumption that every factor has an equal importance value of 10, indicating that each scenario has a full score of 90.The formula used to calculate the summation of factor scores is given below.
, 2) developing AIMSUN simulation model, 3) estimating traffic demand by using dynamic O/D adjustment, 4) using quantitative techniques to statistically validate the model, 5) creating statistical measurement criteria, 6) applying the multi-factor scoring method to carry out O/D matrix selection and adjustment, and 7) obtaining the results of traffic demand estimation.The details are shown in Fig. (2).
Scoring Weight k = scoring weight of factor k, and Score = summation of factor score.