High-performance on-road vehicle detection with non-biased cascade classifier by weight-balanced training
© Cho and Hwang. 2015
Received: 25 February 2015
Accepted: 2 June 2015
Published: 6 June 2015
In this paper, we propose a cascade classifier for high-performance on-road vehicle detection. The proposed system deliberately selects constituent weak classifiers that are expected to show good performance in real detection environments. The weak classifiers selected at a cascade stage using AdaBoost are assessed for their effectiveness in vehicle detection. By applying the selected weak classifiers with their own confidence levels to another set of image samples, the system observes the resultant weights of those samples to assess the biasing of the selected weak classifiers. Once they are estimated as biased toward either positive or negative samples, the weak classifiers are discarded, and the selection process is restarted after adjusting the weights of the training samples. Experimental results show that a cascade classifier using weak classifiers selected by the proposed method has a higher detection performance.
Vehicle detection is a binary classification problem that distinguishes vehicles of different colors and shapes from cluttered backgrounds. Vehicle detection is employed in driver assistance systems and in autonomous vehicles [1, 2]. In surveillance systems that perform object detection using images from a static camera, differential images are used to locate the region of interest (ROI). In on-road detection environments, the background of images captured in a moving vehicle is not fixed and changes continuously. The ROI cannot be identified and the whole region of an image should be searched to detect vehicles. This means that a much larger number of operations is required. In applications related to on-road vehicle and transportation, driver safety is as important as convenience; thus, a robust and real-time detection system should be capable of providing an instant alarm to the driver or system . From the driver’s viewpoint, a late alarm would, in effect, be equivalent to a detection error. The system should give a warning to the driver as early as possible and needs to detect vehicles in the distance. This requires increased image resolution and makes the detection system more complex.
Radar sensors have been used for vehicle detection [1, 3]. They can detect objects without complex computation, even under poor illumination conditions. However, due to interference problems among sensors and poor lateral resolution , passive sensors such as cameras have been commonly employed for vehicle detection. Recently, the part-based model approach proposed by Felzenszwalb et al.  was applied to vehicle detection systems [6–8]. By detecting the parts of vehicles from images, their model combines the detected parts to detect vehicles . While the part-based vehicle model improves detection performance, its computation complexity increases due to its algorithmic structure. Neural networks have emerged as a powerful machine learning model and have shown outstanding performance in detection systems [9, 10]. However, they require a tremendous amount of computing time, which makes it difficult to apply them to the detection of multiple moving objects in real time.
The cascade classifier proposed by Viola and Jones  has been commonly employed for real-time vehicle detection. The cascade classifier achieves both high processing speed and detection performance by employing simple classifiers at early stages to reject non-objects and complex classifiers at later stages. However, the detection rate is decreased as the stage proceeds in the cascade, and there is a large gap between the observed detection rate and the theoretical one. This is caused by biased classifiers. A classifier may be biased by unbalanced training samples due to a disparity between the difficulty of positive samples and that of negative ones. Each stage of a cascade classifier is trained by AdaBoost with training samples prepared using bootstrapping. Bootstrapping collects samples misclassified at a previous stage. It makes negative samples more difficult than positive ones as the stage proceeds; thus, training comes to focus more on negative samples.
Several methods have been proposed to improve the detection performance of cascade classifiers by assigning larger weights to positive samples. Most of these studies assumed that AdaBoost has a faulty weight update and modified it to assign larger weights to positive samples. These works improved detection performance for face detection; however, they failed to provide the weight values to be assigned to the positive samples.
In this paper, a new cascade classifier is proposed, in which the weak classifiers selected at each stage are confirmed for their effectiveness in the detection process using another sample set (reservoir set). The weights of these samples are updated by applying the selected weak classifiers using the same process in AdaBoost. The disparity, between the total weight assigned to the positive and negative samples in the reservoir set, is regarded as the degree of weight unbalance in the positive/negative training samples and bias in the selected weak classifiers. To generate non-biased classifiers, the disparity should be reduced. If the initial weight of the training samples is adjusted prior to training, the unbalance in the training samples is expected to be reduced and the weak classifiers selected after retraining are expected to be less biased. To determine the initial weight, the ratio of total weight assigned to the positive to the negative samples in the reservoir set is used as a reference value. This process is continued at each stage until non-biased weak classifiers are selected.
The rest of this paper is organized as follows: Section 2 presents the AdaBoost algorithm, the conventional cascade classifier, and asymmetric boosting. Section 3 describes the proposed system and its algorithmic flow. Experimental results showing the performance of the proposed system are presented in Section 4. Conclusions are drawn in Section 5.
2 Background and related works
In this section, a background is provided that is required for the understanding of the proposed system. The AdaBoost and conventional cascade learning algorithms are briefly described. Asymmetric boosting is then presented together with related research works.
2.1 AdaBoost and cascade learning
AdaBoost is known to provide a principled and highly efficient mechanism for feature selection [12, 13]. In AdaBoost, a strong classifier is constructed by iterating rounds of weak classifier selection and sample weight updating. The weights are decreased for correctly classified samples and increased for incorrectly classified ones so that AdaBoost can focus on difficult samples [14, 15]. Freund and Schapire proved that the training error of a strong classifier approaches zero exponentially in the number of rounds [16, 17].
where N + and N_ are the numbers of positive and negative samples in X, respectively.
In the conventional cascade classifier, simple classifiers are employed at early stages to reject the negative samples, whereas complex classifiers are used at later stages to achieve a high detection rate [18, 19]. The structure of the cascade reflects the fact that within any single image, the overwhelming majority of sub-windows are negative . The bootstrapping procedure  is employed to collect negative samples in the training cascade. Each stage uses misclassified samples at its precedent stage by bootstrapping its training samples. Later stages are trained by more difficult negative samples, and the classifiers become more complex.
The positive rate (p j ), which is the rate of true and false positives against the number of samples, can be approximated to f j , as positive samples are extremely rare in detection environments .
2.2 Asymmetric boosting
Asymmetric boosting has been proposed by several researchers [12, 17, 21–24]. It assigns larger weights to positive samples than negative ones to overcome the problem that false positives and false negatives are treated without distinction in AdaBoost. According to Viola and Jones, one limitation of AdaBoost arises in the rare occurrence of positive samples . The impact of misclassified positive samples is much higher than that of misclassified negative ones, and rejected positive samples cannot be restored in a cascade classifier. This necessitates that false negatives be minimized. Various researchers have proposed methods to improve the performance of the AdaBoost algorithm through asymmetric weight assignment. The weight update of AdaBoost was modified so that false negatives are assigned larger weight than false positives [12, 17, 21, 22, 24].
Viola and Jones  applied asymmetry to AdaBoost by assigning k times larger weight to false negatives than false positives. Fan et al.  proposed AdaCost, a cost-sensitive extension of AdaBoost, in which false negatives are assigned larger weights than false positives and true negatives are assigned smaller weights than true positives. Ting  proposed a method that assigns larger weights to false negatives based on a cost function that reflects the importance of misclassified samples. While these methods achieve higher detection performance, the weight update is heuristic, making it difficult to predict performance prior to in-field use.
They did not suggest any method that derives an optimal asymmetry parameter of γ for performance improvement.
3 Proposed system
The proposed system takes the form of the conventional cascade. The difference is in the structure and operation of the constituent stages, especially in the training process. Each stage of a conventional cascade performs feature (weak classifier) selection. Additionally, the proposed cascade performs an evaluation of the selected weak classifiers to assess their effectiveness. Although the selected weak classifiers might achieve the performance goal, they were trained using the samples collected by bootstrapping. Therefore, the weak classifiers tend to become biased as unbalance between the positive and negative samples in the training set increases. In detection environments, false negatives are much more critical than false positives, while objects are less frequent than non-objects. Thus, non-biased weak classifiers are desired. The estimated bias in the selected weak classifiers is used to adjust the weights in the training samples. By adjusting the initial weight of the training samples prior to training, the unbalance between positive and negative samples in the training set is reduced, and the weak classifiers selected by using weight-adjusted samples are less biased. In the proposed system, this is handled as follows: By applying weak classifiers to another set of samples, the reservoir set, the weights of these samples are updated with the same process that updates the weights of the samples in training. The degree of weight disparity in the positive and negative samples is estimated with the weight ratio, which is the ratio of the total weight assigned to positive to negative samples. In this research, the weight ratio is used as a reference to estimate to what extent the selected weak classifiers are biased to positive or negative samples. The training is then restarted using the training samples whose weights are adjusted by the weight ratio. If the positive and negative samples in the training set are balanced, the weight ratio converges to 1. This process prevents the detection rate by the selected weak classifiers from decreasing in later stages; thus, the detection performance of the cascade classifier improves.
The training for the corresponding stage is restarted. This process is repeated until the weight ratio converges to 1 or there is no further improvement.
4 Experimental results
To show the effectiveness of the proposed system, a series of experiments were performed. For the positive samples, 26,000 vehicle images were obtained by cropping the images captured on the road with a camera mounted on vehicle. All of the vehicle samples were resized to 20 × 20 pixels. For each stage of the cascades, 6,000 samples were used for training. A half of remaining 20,000 samples were used as the reservoir set, and the other half were used to evaluate the detection performance of the weak classifiers employing the proposed system. Over 500,000 negative samples were prepared by randomly cropping 1,085 background images that did not contain any vehicle objects. These samples were included in the training set, the test set, and the reservoir set.
Vehicle detection systems were trained by three different approaches: the conventional cascade learning approach, the asymmetric boosting approach, and the proposed system. Each stage in the cascade classifiers was trained to achieve a detection rate of 0.998 and a false alarm rate of 0.5. The detection system employing the asymmetric boosting approach was trained by assigning the initial weights to the training samples such that the sum of the positive sample weights is two times and four times greater than that of the negative sample weights. In the proposed system, the weight is assigned by extracting the latent asymmetry at each stage in the cascade classifier. The termination condition of each stage is set to have the weight ratio of 1.
Comparison of number of operations
Number of operations
Comparison w/ Viola&Jones
AsymBoost(W = 2)
AsymBoost(W = 4)
To obtain a high-performance cascade classifier, weak classifiers constituting each stage should be selected considerately, even with increased training time. Weak classifiers not biased toward either positive or negative samples are desired. This is due to the fact that false negatives are much more critical than false positives while objects are less frequent than non-objects in detection environments. The selected weak classifiers in conventional cascade classifiers are biased, and they lead to a gap between performance goal and observed performance at each stage.
In this paper, a cascade classifier is proposed in which the selected weak classifiers are confirmed for their effectiveness in detection process. The main factor contributing to biased weak classifiers is unbalanced training samples prepared by the bootstrapping procedure. To assess the bias of the selected weak classifiers, they are applied to the samples in the reservoir set to update their weights. The disparity between the total weight of positive and negative samples in the reservoir set indicates the degree of unbalance in the positive and negative training samples. If they are found to be unbalanced, the weights of the training samples are adjusted and the weak classifier selection process is restarted. Experimental results confirm the effectiveness of the proposed cascade classifier in vehicle detection by showing an improved performance over conventional ones.
- Z Sun, G Bebis, R Miller, On-road vehicle detection: a review. IEEE Trans. Pattern Anal. Mach. Intell. 28(5), 694–711 (2006)View ArticleGoogle Scholar
- C Papageorgiou, T Poggio, A trainable system for object detection. Int. J. Comput. Vis. 38(1), 15–33 (2000)MATHView ArticleGoogle Scholar
- Z Sun, G Bebis, R Miller, On-road vehicle detection using Gabor filters and support vector machines, in Proceedings of 14th Int. Conf. Digital Signal Processing (IEEE, Santorini, Greece, 2002), pp. 1019–1022Google Scholar
- J Cui, F Liu, Z Li, Z Jia, Vehicle localization using a single camera, in Proceedings of IEEE Intelligent Vehicle Symposium (IEEE, San Diego, CA, 2010), pp. 871–876Google Scholar
- PF Felzenszwalb, RB Girshick, D McAllester, D Ramanan, Object detection with discriminatively trained part based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)View ArticleGoogle Scholar
- Y Li, B Tian, B Li, G Xiong, F Zhu, Vehicle detection with a part-based model for complex traffic conditions, in Proceedings of 14th IEEE Int. Conf. Vehicular Electronics and Safety (IEEE, Dongguan, China, 2013), pp. 110–113Google Scholar
- LC Leon, R Hirata, Vehicle detection using mixture of deformable parts models: static and dynamic camera, in Proceedings of 25th SIBGRAPI Conf. Graphics, Patterns and Images (IEEE, Ouro Preto, Brazil, 2012), pp. 237–244View ArticleGoogle Scholar
- S Sivaraman, MM Trivedi, Real-time vehicle detection using parts at intersections, in Proceedings of 15th Int. IEEE Conf. Intelligent Transportation Systems (IEEE, Aleutian, Alaska, 2012), pp. 1519–1524Google Scholar
- PM Daigavane, PR Bajaj, Vehicle detection and neural network application for vehicle classification, in Proceedings of Int. Conf. Computational Intelligence and Communication Networks (IEEE, Gwalior, India, 2011), pp. 758–762Google Scholar
- O Ludwig, U Nunes, Improving the generalization properties of neural networks: an application to vehicle detection, in Proceedings of 11th Int. IEEE Conf. Intelligent Transportation Systems (IEEE, Beijing, China, 2008), pp. 310–315Google Scholar
- P Viola, M Jones, Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)View ArticleGoogle Scholar
- P Viola, M Jones, Fast and robust classification using asymmetric AdaBoost and a detector cascade, in Proceedings of Advances in Neural Information Processing Systems (NIPS, Vancouver, Canada, 2001), pp. 1311–1318Google Scholar
- Y Freund, RE Schapire, A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Sys. Sci. 55(1), 119–139 (1997)MATHMathSciNetView ArticleGoogle Scholar
- R Duda, P Hart, D Stork, Pattern classification, 2nd edn. (Wiley-Interscience, New York, 2001)MATHGoogle Scholar
- Y Freund, RE Schapire, A short introduction to boosting. J. Jpn. Soc. Artif. Intell. 14(5), 771–780 (1999)Google Scholar
- P Viola, M Jones, Rapid object detection using a boosted cascade of simple features, in Proceedings of IEEE Computer Society Conf. Computer Vision and Pattern Recognition (IEEE, Kauai, HI, 2001), pp. 511–518Google Scholar
- H Masnadi-Shirazi, N Vasconcelos, Asymmetric boosting, in Proceedings of Int. Conf. Machine Learning (ACM, Corvallis, OR, 2007), pp. 609–619Google Scholar
- R Xiao, L Zhu, HJ Zhang, Boosting chain learning for object detection, in Proceedings of IEEE Int. Conf. Computer Vision (IEEE, Nice, France, 2003), pp. 709–715View ArticleGoogle Scholar
- J Wu, SC Brubaker, MD Mullin, JM Rehg, Fast asymmetric learning for cascade face detection. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 369–382 (2008)View ArticleGoogle Scholar
- KK Sung, T Poggio, Example-based learning for view-based human face detection. IEEE Trans. Pattern Anal. Mach. Intell. 20(1), 39–51 (1998)View ArticleGoogle Scholar
- W Fan, SJ Stolfo, J Zhang, PK Chan, AdaCost: misclassification cost-sensitive boosting, in Proceedings of 16th Int. Conf. Machine Learning (ICML, Bled, Slovenia, 1999), pp. 97–105Google Scholar
- KM Ting, A comparative study of cost-sensitive boosting algorithms, in Proceedings of 17th Int. Conf. Machine Learning (ICML, Stanford, CA, 2000), pp. 983–990Google Scholar
- I Landesa-Vázquez, JL Alba-Castro, Shedding light on the asymmetric learning capability of AdaBoost. Pattern Recognit. Lett. 33(3), 247–255 (2012)View ArticleGoogle Scholar
- Y Sun, MS Kamel, AKC Wong, Y Wang, Cost-sensitive boosting for classification of imbalanced data. IEEE Trans. Pattern Anal. Mach. Intell. 40(12), 3358–3378 (2007)MATHGoogle Scholar
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.