High-Occupancy Vehicle Lane Enforcement System

2.1. Passenger Detection Model

Considering the high speed of vehicles on the freeway, a quick detection capability is of paramount importance for a reliable HOV lane enforcement system. Hence, the You Only Look Once (YOLO)-based passenger detection model was established. The YOLO algorithm creates spatially separated bounding boxes and related class probabilities for object detection. It models object detection as a regression problem. It grids the target image and then bounding boxes, confidence for the boxes, and class probabilities are predicted for each grid cell [13].

Unlike other convolutional neural network-based object detection algorithms, it utilizes a single neural network to predict bounding boxes and class probabilities directly on the entire image in one assessment. Therefore, it can be optimized end-to-end directly on detection performance due to the fact that the whole detection pipeline comprises a single network [14]. All these characteristics make YOLO widely recognized for quick object detection. Among YOLO algorithms, YOLOv3 [15], an improvement of the initial YOLO algorithm in terms of methods for bounding box prediction, class prediction, and feature extractor, was chosen for this study.

The procedure for constructing the YOLO model is presented in Fig. (3). The raw data gathered on the road amounted to 2,126 images and these were augmented for building a reliable YOLO model to 6,378 images using widely recognized augmentation techniques, such as geometric transformation, rotation, and noise injection [16]. Individual images obtained from camera 1 (Fig. 4) were segmented by boxing on the basis of the window, the B pillar which divides the first and second rows of seats, and the passenger face, and then each of the divided segments was labeled for the corresponding object detection. Images from camera 3, which only targets the last row of seats, were only segmented based on the occupant's face. The ratio of training and test images was 8:2 and the accuracy of the trained YOLO model with Intersection over Union (IOU) of 50% was approximately 95% (Fig. 5).

2.2. System Components

The developed system is broadly categorized into hardware and software. The hardware, as photographed in Fig. (5), comprises three high-speed (300 frames per second) video infrared cameras, five infrared illuminations, and two lasers. Of the three cameras, one is for recognizing the number plates of violating vehicles and the remaining two are for counting the occupants in the compartment. The infrared illuminations, with the same frequency as the cameras, are also installed near the corresponding cameras. For compartment detection, two illuminations for each camera were used for maintaining the occupant detection capabilities regardless of tinted windows or the intensity of illumination. For triggering the image capture for occupant detection, two laser sensors are employed. The two lasers can also measure the speed of passing vehicles to investigate if the occupant detection capability varies according to the vehicle speed.

The software, as shown in Fig. (6), captures images from the three cameras as described above. It can automatically adjust the brightness, shutter speed, and aperture of each camera according to the external illumination. The detection results are presented in the lower-left side of Fig. (6). For number plate recognition, an open software source for Korean vehicle registration numbers, easily obtained on the Internet, is used, showing a satisfactory outcome of more than 99% accuracy. Using the YOLOv3-based deep learning model, occupant detection is performed. For reliable occupant detection in the first and second rows of seats, the B pillar and window areas are also detected and boxed, whereas, for simplicity, only the occupants of the last row of seats are detected. To transmit the results of the object detection to the police server system (Fig. 2), the TCP/IP protocol suited for instant message transfer, a requirement for the HOV lane enforcement system, was employed.

3.1. Data Collection

To evaluate the system, including the hardware and software described above, field data using two kinds of recreational vehicles that occupy around 80% of HOVs in Korea were obtained on a proving ground where arbitrary speed can be maintained. The data were collected in the day- and night-time on November 25^th, 2020, as photographed in Fig. (7). The vehicles in the experiment, with one to six passengers, were driven at speeds of 20, 40, 60, 80, and 100km/h, respectively. There was no lane changing and straddling maneuvers while driving that might cause errors in detection. The images for evaluation tallied up to 245.

3.2. Numerical Analysis

The confusion matrix according to the number of passengers is presented in Table 1. All of the cases undercount the occupants: the error for six passengers is the highest and the lowest is for one passenger. Specific detection errors by speed and occupants are shown in Table 2 and three-dimensionally plotted in Fig. (8). The average error was 9%, and the highest error of 80% was recorded when vehicles with six passengers were driven at 40km/h. No error was observed for most of the cases. To investigate whether any notable differences exist according to the speed and number of passengers, an analysis of variance (ANOVA) was performed. The result, shown in Table 3, showed that no statistically significant differences were revealed by the ANOVA tests at a 0.05 significance level, as the P-values (0.70 for the speed; 0.33 for the number of passengers) are higher than the significant level. The errors did not vary statistically by illumination. Typical cases of detection error are captured in Fig. (9). The developed system mostly undercounts occupants when face contours are obscured by hair or not clearly visible due to mirrored clouds.

Although no notable difference was identified from the ANOVA test, errors for six passengers showed higher than the other cases. To improve the performance for six passengers, more images under diverse conditions probably need to be obtained for training the YOLO model. Actually, the growing number of passengers requires higher detection capability. However, the images used for training the model were almost equally distributed from one to six passengers.