RESEARCH ARTICLE


Empirical Study on the Accuracy and Precision of Automatic Passenger Counting in European Bus Services



Alessandro Olivo1, Giulio Maternini2, Benedetto Barabino1, 2, *
1 Department of Civil, Environment and Architecture Engineering (DICAAR), University of Cagliari, Cagliari, Italy
2 Department of Civil, Environment, Land and Architecture Engineering and Matematics (DICATAM), University of Brescia, Brescia, Italy
3 Department of Studies and Research, CTM SpA, Cagliari, Italy


© 2019 Olivo et al

open-access license: This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International Public License (CC-BY 4.0), a copy of which is available at: (https://creativecommons.org/licenses/by/4.0/legalcode). This license permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

* Address correspondence to this author at the Department of Civil, Environment, Land and Architecture Engineering and Matematics (DICATAM), University of Brescia, Italy; Tel: +39 0303711306;
E-mail: benedetto.barabino@unibs.it


Abstract

Background:

An Automatic Passenger Counting system represents a powerful resource for an efficient operational planning of public transport companies, but it gives rise to several challenges such as accuracy and precision, which must be addressed in order to operate successfully.

Objective:

Unlike previous studies in the North American bus market, this paper evaluates the accuracy and precision of an infrared APC system in a European bus market.

Methods:

The accuracy is evaluated by considering: (i) the presence/absence of the error and its direction; (ii) the magnitude of the error disregarding the direction and (iii) some tests on the nature of the error. The precision is evaluated by direct and inverse regression models and some t-test on biases.

Results:

As for accuracy, a small average magnitude of the errors is observed. In addition, the APC accurately measures alighting passengers, while it presents a slight tendency to systematically undercount boarding passengers. As for precision, the amount of measurement error due to the APC system exists and, even if it is relatively contained, it is statistically significant for boarding and alighting passengers.

Conclusion:

Although one type of APC system is evaluated only on one bus, it seems quite accurate for recording alighting passengers, whereas a correction factor should be applied for boarding passengers.

Keywords: European buses, Infrared APC, APC’s accuracy, APC’s magnitude Error, APC’s precision, Inverse regression.



1. INTRODUCTION

Passenger volumes represent the most relevant component of the bus transit service, they are pivotal for efficient planning and operation and provide a key measure of effectiveness for Public Transport Companies (PTCs). Knowing passenger volumes in bus transit services enables long-term planning, scheduling of routes, headways and related timetables. Moreover, this knowledge simplifies short-term planning because some buses may be re-assigned to specific routes, when the congested routes and time-periods are identified. Thus, buses are expected to run where and when passengers want them.

Likewise two European Norms on service quality in public transport have been issued [1, 2]. They require that several quality-based parameters should be satisfied for a predefined number (or percentage) of passengers. Therefore, in order to plan the best service, PTCs need detailed data on passenger volumes.

Recently, passenger transportation has been re-organized in many European countries in regional and local integrated networks, controlled by public authorities, that periodically entrust the management of services to PTCs by tendering procedures [3]. For this management, PTCs receive a variable quota of subsidies depending on their performance measured by passenger volumes, distance travelled and so on. Thus, measuring passenger volumes is relevant to determine the subsidy sharing among competing PTCs in the same integrate network.

As a result, estimating passenger volumes is needed for planning operation, quality certifications, and distribution of subsidies among competing PTCs.

Nowadays, Automatic Passenger Counting (APC) systems are usually adopted among vanguard PTCs to collect passenger volume data. However, even if APC systems result in huge amounts of data, they may present several errors that might limit their massive spread among PTCs.

Unlike previous studies in the North American bus market, this paper aims to evaluate the accuracy (i.e., the systematic over or undercounting of passengers related to the true value) and precision (i.e., the distribution of error between the measured and true value of passenger activity) of a commercial infrared APC system in a real European bus market. This evaluation is performed by using about 1,000 pieces of raw data gathered by an Italian PTC.

Addressing these issues is relevant for practitioners, vendors of APC systems and researchers. There is a strong interest in the transit industry to make the use of APC systems widespread, because in deciding to buy these systems, a key element is the evaluation of accuracy and precision. According to [4], few PTCs can afford the research needed to establish the level of systematic overcount or undercount. Moreover, PTCs are more prone to entrust a mature technology rather than acquire a new technology. Vendors need feedback on real tests to understand if their systems are working well. Finally, this study may serve as a benchmark for researchers that evaluate alternative solutions to the issue of people counting.

The remaining paper is organized as follows. Section 2 reviews the measure of passenger volumes, the APC systems and the related accuracy and precision. Moreover, it discusses the gaps to be addressed. Section 3 presents the methods used to measure the accuracy and precision of the tested APC technology. Section 4 reports the results of a real experimentation on an Italian PTC and discusses them in the context of other studies. Finally, Section 5 provides conclusions and research perspectives.

2. LITERATURE REVIEW

2.1. Measurement of Passenger Volumes

Nowadays, most advanced PTCs collect data on passenger volumes by using four methods. These methods are classified in Table 1 according to the possible use of the data collected with and without passenger participation and the adoption of manual or automatic counting strategies. Table 1 is adapted from [5].

Table 1. Measuring passenger volumes.
Counting Strategy With Passenger’s Participation Without Passenger’s Participation
Manual Travel Document Sold Checker
Automatic Automatic Fare Collection Automatic Passenger Counter

On the one hand, passenger volumes may be measured using methods that require the participation of passengers who buy a ticket. They are the so-called ticket dependent methods.

Many worldwide PTCs determine daily passenger volumes by multiplying the Travel Document Sold (TDS) (ticket and passes, of course) by the average the number of potential trips per passenger. Unfortunately, this modus operandi does not provide insights on passenger volumes per route and time periods, when dedicated tickets are not issued. In addition, it is highly problematic to obtain disaggregated space-time passenger volumes from daily TDS data. Moreover, the recent increase in fare media and payment options has reduced the reliability for using the TDS method, even if it is still adopted in many PTCs worldwide.

Automatic Fare Collection (AFC) technologies help calculate space-time passenger volumes by smart cards [6]. Indeed, they record the number of tickets and passes tapped at specific points of the route either off-board (e.g., in turnstiles) or on-board (e.g., in validation ticket machines).

However, all ticket dependent methods have some drawbacks.

First, unlike other transportation systems issuing point-to-point tickets (e.g., airlines, ferries, and railways), these methods do not provide accurate knowledge of passenger volumes. Indeed, in not fully gated transit systems, which do not require passengers to tap-out their tickets before exiting, data on alighting and/or transfer stops are missing and need to be inferred. Moreover, in many worldwide PTCs, pass holders are not required to tap in/out their tickets, thus these passengers may be underestimated. This missing information may be inferred by e.g., Radio-Frequency Identification (RFID) technology to track both the origin and destination of passenger volumes in a passive way e.g., [7]. However, passengers may not have any RFID media.

Second, AFC systems track the access (and/or the exit) to the system, thus they provide atomic data for each passenger. However, as smart cards usually contain personal details, data privacy is a crucial issue to be addressed.

Third, ticket-dependent methods typically underestimate passenger volumes owing to possible fare evaders, who do not buy tickets or have invalid ones [8-10]. Therefore, ticket-dependent methods may not be extremely accurate, if data on passenger volumes are used to rearrange bus service planning and operations.

On the other hand, to overcome these drawbacks, vanguard PTCs also use methods that do not require the passenger’s participation. They are the so-called ticket independent methods.

Due to budget restrictions and lack of technology, data on passenger volumes are usually collected during some time periods by trained checkers. Thus, measurements are performed only at selected checkpoints (e.g., maximum load sections) or for a sample of trips over some hours for few days in a year. When a sample of trips is investigated, it is possible to build a load profile of the route by ride checks [11]. Unfortunately, these activities exhibit a significant level of empiricism and unpredictability because they force PTCs to operate with little, if any, data. Therefore, PTCs are not in an ideal position to revise service planning and operations accurately.

A continuous measurement of passenger volumes over space (at each bus stop) and time (at each period) would improve the characterization of the service and remove shortcomings that derive from pre-selected checkpoints or a sample of trips, and aggregating data over long time periods. This continuous measurement can be performed by APC systems.

2.2. APC System

APC systems are not a new topic for the ITS community and are not emerging technologies. Since the mid-1970s, several USA and Canadian PTCs have implemented APC systems [12-15]. The same applied in Europe, later. A wide range of competing APC systems have been developed. These systems may be classified according to the measurement of passenger volumes that can be indirect or direct.

In the case of an indirect measurement, two main systems exist: weight-based and mobile device-based.

By using the weight-based system, passenger volumes could be estimated by weighing all on board passengers by load sensors on the ground or on the suspensions or on the braking system [16, 17]. For instance, Nielsen et al. [17] analysed a new counting technique that exploits electronic weighing systems to control braking in rail systems. In this case, passenger volumes can be estimated by the total weight of passengers on the train.

By using the mobile device-based system, passenger volumes could be measured by counting their portable devices (e.g., phones and smartphones, tablets, and smartwatches). More precisely, the existing literature includes: (i) large-scale cell phone systems [18], (ii) Smartphone apps-based systems [19, 20] and (iii) Wi-Fi systems [21-23]. Systems (i) help collect data once the device is connected to the cellular network (e.g., a call made or received, a short message sent or received, a connection to the internet). Systems (ii) help track voluntary passengers or estimate the passenger volumes onboard by the participation of passengers. Systems (iii) help collect data once the device has an active Wi-Fi interface, regardless of the connection of the owner to a network.

However, all indirect APC systems present drawbacks limiting their successful application in bus transit services.

Weight-based systems provide the total weight of passengers on board but do not offer data on the flow of passengers. Therefore, if a bus weighs the same before and after a bus stop, this could be either due to no boarding or alighting passengers or the quasi-same number of boarding and alighting passengers. Thus, it is necessary to complete the electronic weighing equipment with a method to gather passenger volume information.

Mobile device-based systems present several pitfalls as well. Children or other people may not have mobile devices, or some passengers may carry on more than one device, thus, passenger volumes may be underestimated or overestimated. Systems (i) and (ii) need the collaboration with telco operators and/or the consent of passengers to estimate passenger volumes. Besides, as for system (ii), passenger volumes severely depend on the willingness of passengers to participate, and, this fact may not be compatible with APC systems that rely on crowd counting principles. Finally, due to the randomization of MAC addresses [24], tracking devices are no longer feasible using system (iii) [25].

In the case of a direct measurement, passenger volumes could be estimated by recognizing people when they board or alight the bus. Three main systems include old mats technologies (pressure sensitive or multiswitch), consolidated infrared technologies (passive or active) and recent video image technologies [13-16, 26].

For instance, multiswitch mats technologies measure passenger volumes by the patterns of footstep changes.

Infrared technologies measure passenger volumes by light beams. When the beams are interrupted, a count is registered. The sequence in which the beams are broken determines the direction of the movement of the passenger. To the authors’ knowledge, infrared technologies are the most adopted in buses and easily found in commerce.

Video image technologies measure passenger volumes by proper cameras in the bus that recognise the passenger. They use several algorithms to a) detect motion, b) estimate its direction, and c) validate the existence of a moving passenger. For instance, Chen et al. [26] used a zenithal camera to capture passenger volumes and the single image is divided in blocks classified according to its motion vector to distinguish between boarding and alighting passengers.

Nevertheless, all “direct” technologies present drawbacks as well. For instance, because mats need to be installed in/on the upper and lower steps of a stairwell, they cannot be adopted in recent low floor bus configurations, as it is not possible to differentiate if a passenger boards or alights the bus. Although one could argue that passengers can be distinguished by specialized doors, this operation is not appreciated by the bus drivers or the passengers. Passengers prefer to board/alight at the most convenient door (i.e., the closest to their waiting point); bus drivers need to accurately approach the bus stop area. A further drawback might be the capital cost due to the need to install at least more than one sensor per doorway.

However, apart from mats, the remaining direct APC systems appear easily applicable to recent transit services, as they can usually be installed on any kind of vehicle. As follows, the authors refer to these systems.

2.3. Accuracy and Precision of APC Systems

Usually, before using APC data in the operational management, two crucial factors are evaluated: the accuracy and precision.

The accuracy of raw count tends to be the focus of vendors and many PTCs. Its analysis evaluates the capability of APC systems to count passengers well. The accuracy measures the systematic over or undercounting of passenger volumes relative to the “ground truth”. Indeed, if the estimation of passenger volumes statistically differs from the ground truth, it is inaccurate. Thus, if the systematic error is known, correction factors need to be calculated to account for the biases.

The precision measures the distribution of errors between the measured and true value of passenger activity. The precision is always evaluated according to a level of confidence. For instance, a PTC would be 90% certain that passenger volumes collected by APC systems fall within ±5% of their true value.

Although accuracy and precision refer to errors in the measurement, the fundamental difference is on the nature of the error. Indeed, accuracy refers to systematic errors, whereas precision refers to random errors. Thus, unlike accuracy, a correction factor cannot be applied for low precision measurements.

Table 2 reports the outcomes of some applied research on direct APC systems largely focused on mats and infrared technologies on buses, because these technologies are among the most adopted in the real operation of buses, according to the authors’ knowledge.

A sample of boarding and alighting passenger data collected at bus stops is considered in order to measure the accuracy and precision. Moreover, unlike [32] and [33] that adopted video surveillance cameras as a comparison method, the evaluation of accuracy was done by comparing, at each stop, APC data with manual data taken by checkers on the bus (manual data are supposed to be error free).

Simple methods evaluated the accuracy by measuring the error rate of counting 1 for boarding and alighting passengers or the error rate of a range of boarding/alighting conditions on routes exhibiting from 1 to 12 passengers at any given stop by cross-tabulation. More refined methods largely adopted confidence intervals and mean tests for paired data in order to evaluate if systematic errors existed. Other studies refined the accuracy by the Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) to evaluate the magnitude of the error [5].

As for precision, direct and inverse regression models were adopted to test the relative effect of measurement error as suggested by [35, 36].

In general, these studies revealed that APC systems undercounted passenger activity and were more accurate when they recorded boarding rather than alighting passengers. Moreover, even if discrepancies existed between manual counting and APCs, they were often not significant.

Studies [32, 33] showed that the amount of measurement error due to an APC system was statistically significant (at a 95% level of confidence) for boarding and alighting passengers and loads in all samples of buses examined. Moreover, the amount of estimation bias related to load estimates attributed to the APC system is 8.8% using passenger volumes obtained from video cameras as opposed to 26.8% obtained from manual counting.

2.4. Gaps in the Literature

Although all these studies added relevant contributions, some considerations arose.

First, even if it is common to use APC data for many purposes, to the best of the authors’ knowledge, research focusing on on-field evaluation of accuracy and precision of APC systems is largely found for American bus configurations, that differ from European ones. Unlike the American bus configurations, in Europe, buses are usually designed to accommodate many standing instead of seated passengers. Moreover, other differences may be observed owing to the location of sensors, the behaviour of passengers next to doors (e.g., passenger bunching) and so on. Thus, these differences might affect the analysis of accuracy and precision.

Second, even if the accuracy and the magnitude of the counting error were partially evaluated [5, 32, 33] were the only studies that assessed the precision.

Third, vendors often evaluate their products and usually provide data accuracy statistics. However, the accuracy rates provided by vendors tend to be higher than actual rates because they usually work on ideal conditions. Thus, their statistics is usually not based on the real operation of buses, which are affected by situations that are different from the ideal ones. For instance, sensors can be dirty, thus they may result in an inaccurate measurement of passenger activity.

Therefore, it is worth shedding light on the evaluation of accuracy and precision of APC systems also in a European bus market.

3. MATERIALS AND METHODS

3.1. Experimental Setup

The experimentation was performed in the metropolitan area of Cagliari (Sardinia, Italy) having about 0.4 million inhabitants. The main local PTC, namely CTM, manages the public transportation with 271 vehicles (i.e., buses and trolleys) and serves approximately 38.9 million trips a year. Moreover, these vehicles travel over 12,3 million kilometres per year along 32 urban routes [37].

Nowadays, CTM collects passenger data by TDS and manual counting. However, because managers and planners have little trust in manually collected data, CTM is motivated to use APC systems. Moreover, there is a further motivation: their routes are certified according to EN 13816:2002 and are required to express measurements in passenger volumes [38-40].


Table 2. Accuracy and precision of experimentations of some direct APC technologies.
Authors Location APC Technology Transit mode Vehicles Tested [#] Boarding/alighting stop-level data [#] Accuracy (Precision) evaluation method Accuracy (Precision) main results
Deibel and Wood [27] USA I, U, P, M and A Bus 2 n/a Error rate of counting (n/a) Generally, APC undercount boarding and alighting passengers; M provided an accuracy better than 98 % of passengers’ volumes; however, the maximum load is significantly less than I;
Poirier and Hobbs [28] USA I and M Bus 9 8,600 Cross tabulation of error rate of counting errors for classes (n/a) APCs are accurate i.e., no count’s errors, for 79% of the observations
Attanuccci and Vozzolo [13] USA I and M Bus n/a n/a Review of 12 case studies (n/a) M APCs are accurate, i.e., no count’s errors, for about 91.5% of the observations; Undercount boarding and alighting passengers; Boarding more accurate than alighting
Strathman [29]; [30] and Strathman and Hopper [31] USA I Bus 46 3,768 Confidence intervals and mean test (n/a) Overcount boarding and alighting passengers with statistical accuracy
Kimpel et al. [32, 33] USA I Bus n/a 2,921 Error rate of counting, Confidence intervals and mean test (Direct and Reverse regression models and t-test) Overcount boarding, alighting and loads; Boarding and alighting passengers are measured with statistical accuracy for low-floor buses. (The amount of measurement error due to APC is statistically significant)
Strathman et al. [34] USA I Rail n/a 722 Confidence intervals and mean test (n/a) Undercount boarding and overcount alighting passengers
Barabino et al. [5] Europe I Bus 1 950 Confidence intervals and mean test, mean absolute error, Root Mean Squared Error; (n/a) Undercount boarding and alighting, unlike boarding, alighting passengers are measured with statistical accuracy; low magnate errors
Representative, but not comprehensive references’ list related. APC Technology: I= Infrared beam; U = Ultrasonic beam; P = Pressure sensitive mat; M = Multi-switch treadle; A = Acoustic echo ranging system; EWE = Electronic Weighing Equipment.

A commercial infrared technology was tested. This technology uses active infrared sensors with beams in the invisible infrared spectrum. These sensors consist of a transmitter, which projects at least two beams vertically across the bus doorways to a light sensitive receiver. The sensors are installed unobtrusively over the vehicle doors and deliver passenger count data. A local unit mainly records data on boarding and alighting passengers, as well as the dwell and departure times of the bus at each bus stop. Moreover, this APC system includes a GPS module. As a result, because the GPS coordinates are available, the location data are matched to the APC data. At the end of each counting session, data are transmitted by a wireless connection to the data management system. Daily data are stored in a central database.

Evaluation tests were carried out on the most traveled high-frequency route of CTM to account for the worst operational conditions. This route presents the following characteristics:

  • Their headways range from 5 minutes to 10 minutes, from 07.00 AM to 8.00 PM.
  • The vehicles deployed have one typology.
  • It is close to regional government offices, schools, hospitals, and shopping malls.

Due to budget constraints, only one bus was equipped with this technology.

CTM tested this APC system at the end of 20102.

Table 3 reports some characteristics of the bus selected for the experimentation. Table 3 is self-explanatory. Since, the number of counting sensors depends on the number of bus doors, 3 infrared sensors were installed in the vehicle.

Table 3. Bus characteristics.
APC Number Manufacturer Model Length [m] Doors [#] Capacity [Pass.] Bus Type
Active Infrared 531 Iveco Citelis 11.990 3 101 Low floor

Some arrangements were performed before collecting and analysing data.

The way bus drivers and checkers are treated is a relevant issue to consider when discussing the accuracy, because the APC system does not distinguish between bus drivers and passengers. Thus, the bus driver was included in boarding and alighting estimates as well as the checkers.

Second, no counting method helps provide 100% accuracy for any operational conditions, either automatic or manual. However, in this paper, manual counting is supposed to be error-free, even if this hypothesis might be doubtful [15, 32, 33]. Nevertheless, to limit this fact, checkers were accurately trained and one checker per bus door was adopted. More precisely, on-board checkers retrieved manual data as follows: Three checkers functioned as a team on the bus. Each checker monitored and recorded boarding and alighting passengers at each door, respectively. Moreover, the checker was responsible for recording his/her own observations and comments. At the end of each ride, data were merged to present the total boarding and alighting passengers at each bus stop. To summarize, the manual counting represented the ground truth.

A total of 950 stop-level data of boarding and alighting passengers on 24 rides over 3 weekdays were collected.

3.2. Evaluation Methods

Selected variables for the analysis are presented in Table 4. Passenger activity variables generated from manual data include boarding passengers (In) and alighting passengers (Out). Passenger activity variables generated from APC data are boarding passengers (APC_In) and alighting passengers (APC_Out). Finally, variables representing the difference between APC data and manual data are boarding differences (Diff_APC_In) and alighting differences (Diff_APC_Out), respectively.

Table 4. Variables in the study.
Variable Description
In Boarding passengers (Manual)
APC_In Boarding passengers (Infrared)
Out Alighting passengers (Manual)
APC_Out Alighting passengers (Infrared)
Diff_APC_In Boarding passenger difference (Infrared – Manual)
Diff_APC_Out Alighting passenger difference (Infrared – Manual)


Unlike [32] and [33], the variable load (i.e., accumulated boarding passengers minus accumulated alighting passengers) was not considered, for two reasons.

First, according to [5], in order to analyse load, proper processing and parsing is needed at vehicle-block and ride levels as well as for the correction of negative loads. Thus, only raw data (i.e., not ‘processed’) are considered.

Second, loads need to be inferred from raw counts. Thus, if errors are detected in raw data, further errors occur in loads, which can be of larger size. Moreover, inaccurate computations of loads may occur due to “drift” effect and may result in a significant overestimation of passenger volumes after many rides, if it is not correctly addressed [4, 5]. However, because raw data are considered, this effect may be neglected. Thus, whether boarding and alighting passenger volumes are accurate, the load estimation might be considered accurate as well.

3.2.1. Evaluation of the Accuracy

To evaluate the accuracy, a hierarchical approach was proposed: (i) the analysis of the presence/absence of the error and its direction; (ii) the analysis of the magnitude of the error, and (iii) the analysis of the nature of the error (i.e., random or systematic).

First, the difference between manual data and APC data was computed, for both boarding and alighting passengers, respectively. Next, the Total Error on Passenger Counts (TEPC), the Average Error for Counts of Boarding (AECB) and the Average Error for Counts of Alighting (AECA) were computed. Let:

  • k be the index of the number of rides;
  • j be the index of the number of bus doors;
  • i be the index of the number of bus stops;
  • r be the number of observed rides;
  • d be the number of observed doors;
  • n be the number of observed bus stops.

The computations of TEPC, AECB and AECA are as follows:

(1)
(2)
(3)

Next, the error rates were computed for classes of boarding and alighting passengers, respectively.

Second, a measure of the average magnitude of the errors was made in the set of APC data without considering its direction, to understand how large errors are. Thus, the MAE and the RMSE were calculated, for both boarding and alighting passengers, respectively. Let:

  • Diff_APC_Ini and Diff_APC_Outi be the difference in boarding and alighting passengers at bus stop i between data returned by the APC system and manual counting, respectively.

The computations of MAEb and RMSEb for boarding passengers and MAEa and RMSEa for alighting passengers are as follows:

(4)
(5)
(6)
(7)

Since they are negatively oriented scores, lower values are better. In our experimentation, APC data were taken as the forecast, while manual data were taken as the observed.

Third, according to [32] and [33], the evaluation of the accuracy was refined with mean tests and confidence intervals to understand the nature of the errors. In fact, even if differences were observed between APC data and manual data, one must understand if these differences are statistically significant.

Means tests were performed for the passenger activity variables expressed as differences between APC data and manual data. APC data are assumed to be accurate if the selected confidence intervals for boarding and alighting passengers encompass 0, respectively. The confidence intervals are denoted by Conf_Intb and Conf_Inta. respectively.

  • Diff_APC_In and Diff_APC_Out be the average value of the difference of boarding passengers and alighting passengers measured automatically and manually, respectively;
  • σb , σa the standard deviation of Diff_APC_In and Diff_APC_Out respectively;
  • α be the significance level (i.e., the probability of rejecting the null hypothesis given that it is true);
  • z be the z-score at the selected confidence interval.

The confidence intervals were computed as follows:

(8)
(9)

In the case of systematic undercounting or overcounting, the related correction factors for boarding and alighting passengers were computed as follows:

(10)
(11)

3.2.2. Evaluation of the Precision

The analysis of the precision was performed as follows.

First, two linear regression models were applied as in [32] and [33]. In the first model, APC data represented the explanatory variable and manual data represented the response variable. The opposite condition occurs for the second model.

Let:

In and Out be the variables of manual counting for boarding and alighting passengers, respectively.

APC_In, APC_Out be the variables of APC data for boarding and alighting passengers, respectively.

α1234 βInOut, βAPC_Out and γ1234 be the intercepts, the regression coefficients and the stochastic noises of each model.

The regression models were formulated as follows, for boarding and alighting passengers, respectively.

(12)
(13)
(14)
(15)

Second, some t-tests at level of significance α determined whether the measurement error associated with the APC system was significantly different from 0 at a selected level of confidence. The tested variables were calculated for boarding and alighting passengers, respectively. The following formulas were adopted, which were taken from [32] and [33].

(16)
(17)

Third, according to [32] and [33], the amounts of estimation bias attributed to the APC system for boarding and alighting passengers were computed as follows:

(18)
(19)

4. RESULTS AND DISCUSSION

Descriptive statistics of the considered data panel is reported in Table 5. Here, for selected variables, the rows show the sample size at stop level (N), the mean values (Mean) and the standard deviation (St. Dev.) as well as the minimum and the maximum values within data. Table 5. is self-explicative.

4.1. APC System Accuracy Results

Values of TEPC, AECB and AECA were computed according to eqns. (1), (2) and (3), respectively. These results are reported in Table 6. Here, one notices that the APC system shows a slight tendency to undercount the total passengers and to undercount both boarding and alighting passengers, respectively. This is because all the percentage differences are negative.

These results contrast [29-33], where APC systems overcount passengers. In addition, the tendency to undercount is stronger for alighting passengers than for boarding ones. Furthermore, these results contrast the hypothesis that passengers board in a more orderly fashion (i.e., one passenger at time) than when they alight [41]. Conversely, our results (i.e., counts are more accurate for alighting passengers than boarding ones) confirms the hypothesis that people tend to rush more when boarding than when alighting a vehicle in a European bus market according to [3].

Our results are partially consistent with [34], that showed that an APC system undercounted boarding passengers and overcounted alighting passengers. However, this last evaluation was performed on rail vehicles, which have a different configuration with respect to buses (i.e., they have wider doors than buses allow simultaneous boarding and alighting movements).

Results of a cross-tabulation analysis of error rates for classes of boarding/alighting passengers are reported in Table 7.

They show that the APC performance achieved is good. Indeed, the APC system returns the truth for about 77% of boarding and alighting passengers, respectively. Even better, these percentages largely increase. if a variance of ±1 passenger is included: the weighted averages provided 95% accuracy for boarding passengers and 97% for alighting passengers, respectively.

Next, the MAE, RMSE and the related difference were computed according to eqns. from (4) to (7) for both boarding and alighting passengers, respectively. These results are reported in Table 8.

Here, one notices that: (i) low values occurred; (ii) RMSE differs from MAE: therefore, all errors are not of the same magnitude. Moreover, (iii) because RMSE is always greater than MAE, there is some variation in the magnitude of the errors; and (iv) because the difference between RMSE-MAE is not so large, large errors are unlikely to have occurred.

As shown in Table 6, there are differences between APC data and manual data. However, the key question is to understand whether these differences are significant. Indeed, if no significant difference is found between APC data and manual data, only random errors occur, and the APC system is assumed to be accurate (or partially). Therefore, according to eqns. (8) and (9), the 95% Conf_Intb and Conf_Inta (p-value <0.05) were computed, respectively. These results are reported in Table 9 .

Here, results indicate that the APC system measures alighting passengers with statistical accuracy; conversely, the APC system systematically undercounts boarding passengers. The result of no statistically significant difference between APC data and manual data for alighting passengers contrasts [31-33], whereas it is consistent with [29] and [30]. This result is also consistent with [32] and [33] when low floor buses are analysed only.

Conversely, the result of a statistically significant difference for boarding passengers contrasts many studies of the American bus market ([29], [30], [31], [32] and [33]). Nevertheless, this result is consistent with [34] that showed that infrared technology significantly undercounts boarding passengers, even if on rail transit.

Moreover, the best accuracy for alighting passengers instead of boarding ones contrasts the hypothesis that standing passengers may obstruct the ride checkers view of the rear door where passengers are expected to alight [28].

According to Table 9, a calibration factor is required for boarding passengers only. This factor was calculated using eqn. (10) and results as 1.0494. Thus, this value should be applied to the boarding passengers collected by the APC system.

4.2. APC System Precision Results

The last analysis presents the results of precisions. They were computed according to eqns. from (12) to (19), respectively. These results are reported in Table 10.

Here, one notices that: (i) according to [32] and [33], the t-tests show that the amount of measurement error due to the APC system is statistically significant for both boarding and alighting passengers; (ii) unlike [32] and [33] for the case of low-floor buses, alighting passengers are shown to have higher measurement errors (i.e., the bias) than boarding passengers; (iii) the highest measurement error regards alighting passengers.

Finally, the relative amount of measurement error-related estimation bias owing to the APC system by using data collected by ride-checkers (i.e., the ground truth) is about 5% for boarding passengers and about 8% for alighting passengers, respectively. These results are consistent with [32] and [33] when low floor buses are analysed only. All studies observed that the amount of estimation biases is significant for both boarding and alighting passengers. However, the present study differs because: (i) manually collected data were compared instead of video surveillance camera data and (ii) the largest bias is reported for alighting passengers instead of boarding passengers.

Table 5. Descriptive statistics.
Variable N Mean St. Dev. Min. Value Max. Value
In 950 1.791 2.970 0 37
Out 950 1.773 2.484 0 21
APC_In 950 1.702 2.863 0 31
APC_Out 950 1.748 2.441 0 17
Diff_APC_In 950 -0.088 0.675 0 6
Diff_APC_Out 950 -0.024 0.722 0 21
Table 6. APC system error rate vs Manual counting.
Variable Difference (%)
TEPC -3,16%
AECB -4,94%
AECA -1,37%
Table 7. Cross tabulation of APC system error rate vs Manual counting.
Boarding Alighting Passengers Number of In Observations % of Time In_APC Provided no Count Errors % of Time In_APC Count Errors was Within ± 1 Passenger Number of Out Observations % of Time Out_APC Provided no Count Errors % of Time Out_APC Count Errors was Within ± 1 passenger
0 456 97.81% 99.78% 406 96.55% 98.77%
1 146 79.45% 99.32% 170 71.18% 97.65%
2 97 63.92% 94.85% 108 64.81% 95.37%
3 92 55.43% 89.13% 88 65.91% 95.45%
4 38 34.21% 86.84% 72 52.78% 100.00%
5 29 24.14% 86.21% 45 28.89% 91.11%
6 38 42.11% 78.95% 19 47.37% 100.00%
7 11 45.45% 90.91% 15 60.00% 80.00%
8 17 29.41% 82.35% 9 33.33% 100.00%
9 7 42.86% 100.00% 4 25.00% 75.00%
10 6 16.67% 66.67% 3 66.67% 100.00%
11 1 0.00% 0.00% 2 100.00% 100.00%
12 6 16.67% 50.00% 1 0.00% 0.00%
Table 8. APC system accuracy. Variation of error.
Variable Values
MAEb 0.29
MAEa 0.29
RMSEb 0.68
RMSEa 0.72
RMSEb - MAEb 0.39
RMSEa - MAEa 0.43
Table 9. APC system accuracy direction of error.
Variable Full sample: 95% Confidence Interval
Diff_APC_In -0.13 -0.05
Diff_APC_Out -0.07 0.02
Table 10. APC system Precision – Measurement error analysis.
Models N βAPC_In βAPC_Out 1/βin; 1/βout Tb(0.05), Ta(0.05) Biasb, Biasa
In, APC_In 950 1.010 - 1.065 - -0.055* - 5.16% -
Out, APC_Out 950 - 0.974 - 1.063 - -0.089* - 8.38%
*It indicates statistically significance (p-value <0.001); the ‘- ‘indicates no available datum.

CONCLUSION

PTCs require having a thorough knowledge of passenger volumes in their vehicles. This is to improve the operational planning, achieve quality service certification and distribute subsidies among competing PTCs. APC systems provide a larger amount of data than human collected data. If APC systems are accurate and precise, related data help provide relevant insights into passenger volumes for each route in order to operate successfully.

Although many studies evaluated the accuracy and precision in the North American bus market, this paper performed these evaluations in a European bus market by using an Infrared APC technology.

As for accuracy, the considered APC system showed: (i) a slight tendency to undercount both boarding and alighting passengers; (ii) a small average magnitude of the errors in the data; (iii) an accurate measure of alighting passengers and a slight tendency to systematically undercount boarding passenger.

As for precision, the amount of measurement error due to the APC system occurred, and it was statistically significant for both boarding and alighting passengers. However, this error was relatively small.

Although this research was small in scale (one APC technology and one bus), it was large enough to evaluate accuracy and precision of a recent APC system in the European bus market.

Future research will provide more solid results if more buses are equipped with this technology. Moreover, the application of this analysis considering additional (and more recent) APC technologies is of great interest. Finally, each APC technology presents its limitations. Therefore, more technologies may be integrated (e.g., RFID, Bluetooth equipment) as redundant tools to overcome these limitations, as already applied in the case of automatic vehicle identification (AVI) technology as in Wiseman ([42]3).

These research topics will have great relevance for future Smart Cities.

LIST OF ABBREVIATIONS

AECA = Average Error for Counts of Alighting
AECB = Average Error for Counts of Boarding
AFC = Automatic Fare Collection
APC Automatic Passenger Counting
AVI Automatic Vehicle Identification
ITS Intelligent Transport Systems
MAC Media Access Control
MAE Mean Absolute Error
PTC Public Transport Company
RFID Radio-Frequency Identification
RMSE Root Mean Squared Error
TDS Travel Document Sold
TEPC Total Error on Passenger Counts

CONSENT FOR PUBLICATION

Not applicable.

AVAILABILITY OF DATA AND MATERIALS

Not applicable.

FUNDING

This work was partially supported by the Italian Ministry of University and Research (MIUR), within the Smart City framework (project: PON04a2_00381 “CAGLIARI2020”).

CONFLICT OF INTEREST

The authors declare no conflict of interest, financial or otherwise.

NOTES

The percentage of times that the APC system returned the same numbers of passengers manually counted and/or with a variance of ± passenger.

Owing to the confidentiality policy of CTM, we are not allowed to use more recent data.

AVI differs from APC owing to the object of the observation (vehicle vs passenger) and the main function performed (enforcing the payment for using the transportation infrastructure and/or detecting wrong driver behaviours vs passenger demand analysis regardless of the payment of fares). However, the idea of the integration among several technologies is quite similar, even if a bit expensive for many PTCs worldwide.

ACKNOWLEDGEMENTS

The Authors are very grateful to the CTM senior management for its support in this work and the opportunity to illustrate the results.

REFERENCES

[1] CEN/TC 320. ‘Transportation – Logistics and services. European Standard EN 13816: Public passenger transport – Service quality definition, targeting and measurement 2002.
[2] CEN/TC 320. ‘Transportation – Logistics and services. European Standard EN 15140: Public passenger transport – Basic requirements and recommendation for systems that measure delivered service quality 2006.
[3] Schwandt H, Baerwolff G. 2010; Validation of Automatic Passenger Counting in Public Transportation Systems Proceedings of the Ninth International Conference on Information and Management Sciences 1-9..
[4] Furth PG, Hemily B, Muller T, Strathman J. Using Archived AVL-APC Data to Improve Transit Performance and Management 2006. TRCP 113.
[5] Barabino B, Di Francesco M, Mozzoni S. An offline framework for handling automatic passenger counting raw data. IEEE Trans Intell Transp Syst 2014; 15(6): 2443-56.
[6] Pelletier MP, Trépanier M, Morency C. Smart card data use in public transit. A literature review. Transp Res, Part C Emerg Technol 2011; 19: 557-68.
[7] Oberli C, Torres-Torriti M, Landau D. Perfomance evaluation of UHF RFID technologies for real-time passenger recognition in intelligent public transportation systems. IEEE Trans Intell Transp Syst 2010; 11(3): 748-53.
[8] Barabino B, Salis S. Moving towards a more accurate level of inspection against fare evasion in proof-of-payment transit systems. Netw Spat Econ in press.
[9] Barabino B, Salis S, Useli B. Fare evasion in proof-of-payment transit systems: Deriving the optimum inspection level. Transp Res, Part B: Methodol 2014; 70: 1-17.
[10] Sasaki Y. Optimal choices of fare collection systems for public transportations: Barrier versus barrier-free. Transp Res, Part B: Methodol 2014; 60: 107-14.
[11] Ceder A. Public Transit Planning and Operation, Theory, Modelling and Practice 2007.
[12] Moore JE, Giuliano G, March JG. Performance of automatic passenger counters in the ventura/lompoc smart card demonstration project. ITS Journal-Intelligent Transportation Systems Journal 2002; 7(2): 131-50.
[13] Attanucci J, Vozzolo D. Assessment of operational effectiveness, accuracy, and costs of automatic passenger counters. Transp Res Rec 1983; (947): 15-23.
[14] Furth PG, Hemily B, Muller THJ, Strathman JG. 2003.Uses of archived avl–apc data to improve transit performance and management: Review and potential. TCRP Web Document 23
[15] Boyle DB. Passenger Counting Systems - A Synthesis of Transit Practice 2008. Tech. Rep. TCRP Synthesis 77.
[16] Pinna I, Dalla Chiara B, Deflorio F. Automatic passenger counting and vehicle load monitoring. Ingegneria Ferroviaria 2010; 65(2): 101-38.
[17] Nielsen BF, Frølich L, Nielsen OA, Filges D. Estimating passenger numbers in trains using existing weighing capabilities. Transportmetrica A: Transp Sci 2014; 10(6): 502-17.
[18] Demissie MG, Phithakkitnukoon S, Sukhvibul T, Antunes F, Gomes R, Bento C. Inferring passenger travel demand to improve urban mobility in developing countries using cell phone data: A case study of Senegal. IEEE Trans Intell Transp Syst 2016; 17(9): 2466-78.
[19] Carrel A, Lau PS, Mishalani RG, Sengupta R, Walker JL. Quantifying transit travel experiences from the users’ perspective with high-resolution smartphone and vehicle location data: Methodologies, validation, and example analyses. Transp Res, Part C Emerg Technol 2015; 58: 224-39.
[20] Chaudhary M, Bansal A, Bansal D, Raman B, Ramakrishnan KK, Aggarwal N. Finding occupancy in buses using crowdsourced data from smartphones. Proceedings of the 17th International Conference on Distributed Computing and Networking 35-39..2016;
[21] Liu H, Yang J, Sidhom S, Wang Y, Chen Y, Ye F. Accurate wifi based localization for smartphones using peer assistance. IEEE Trans Mobile Comput 2013; 13(10): 2199-214.
[22] Kholkine L, Santos PM, Cardote A, Aguiar A. Detecting relative position of user devices and mobile access points. Vehicular Networking Conference (VNC) 2016; 1-8.
[23] Santos PM, Kholkine L, Cardote A, Aguiar A. Context classifier for position-based user association control in vehicular hotspots. Comput Commun 2018; 121: 71-82.
[24] Myrvoll TA, Håkegård JE, Matsui T, Septier F. 2017.Counting public transport passenger using WiFi signatures of mobile devices
[25] Transport for London. Review of the TfL WiFi pilot http://content. tfl.gov.uk/review-tfl-wifi-pilot.pdf
[26] Chen CH, Chang YC, Chen TY, Wang DJ. 2008.People counting system for getting in/out of a bus based on video processing. In Intelligent Systems Design and Applications
[27] Deibel LE, Wood P. ‘On-board data collection for transit operations’. IEEE Trans Vehicular Technol 1977; 26(1): 38-42.
[28] Poirier P J, Hobbs V J. Automated passenger counting systems. synopsis of working group meeting. Synopsis of Working Group Meeting’ 1983. Technical Report (No. DOT-TSC-UMTA-83-23)
[29] Strathman J G. 1989.An evaluation of automatic passenger counters: validation, sampling, and statistical inference. Center for Urban Studies Publications and Reports Paper 109
[30] Strathman JG. 1990.An evaluation of automatic passenger counters: validation, sampling and statistical inference. Final Report (No TNW90-19)
[31] Strathman JG, Hopper JR. Evaluation of automatic passenger counters: validation, sampling, and statistical inference. Transp Res Rec 1991; (1308): 69-77.
[32] Kimpel TJ, Strathman JG, Griffin D, Callas S, Gerhart R. 2002; Automatic passenger counter evaluation: Implications for national transit database reporting 61
[33] Kimpel TJ, Strathman JG, Griffin D, Callas S, Gerhart RL. Automatic passenger counter evaluation: Implications for national transit database reporting. Transp Res Rec 2003; 1835(1): 93-100.
[34] Strathman JG, Kimpel TJ, Callas S. Validation and sampling of automatic rail passenger counters for national transit database and internal reporting at TriMet. Transp Res Rec 2005; 1927(1): 217-22.
[35] Krutchkoff RG. Classical and inverse regression methods of calibration. Technometrics 1967; 9(3): 425-39.
[36] Wonnacott RJ, Wonnacott TH. Econometrics 1970.
[37] CTM. 2018.Carta della mobilità 2017-2018 http://www.ctmcagliari.it/
[38] Barabino B, Di Francesco M. Characterizing, measuring and managing transit service quality. J Adv Transp 2016; 50(5): 818-40.
[39] Barabino B. Automatic recognition of low-quality vehicles and bus stops in bus services. Public Transp (Berl) 2018; 10(2): 257-89.
[40] Barabino B, Lai C, Casari C, Demontis R, Mozzoni S. Rethinking transit time reliability by integrating Automatic Vehicle Location data, passenger patterns and web tools. IEEE Trans Intell Transp Syst 2017; 18(4): 756-66.
[41] Baltes MR, Rey JR. ‘The “Ins and Outs” of APCs: An Overview of Automatic Passenger Counters’. J Public Transp 1999; 2(2): 3.
[42] Wiseman Y. Vehicle Identification by OCR, RFID and Bluetooth for Toll Roads. International Journal of Control and Automation 2018; 11(9): 67-76.