No Access

Illumination Invariance Adaptive Sidewalk Detection Based on Unsupervised Feature Learning

College of Electrical Engineering, Yanshan University, Qinhuangdao, Hebei 066004, P. R. China

E-mail Address: ysu_Wangzy1997@163.com

College of Electrical Engineering, Engineering Research, Center of Intelligent Control System and Intelligent Equipment Ministry of Education, Key Laboratory of Intelligent Rehabilitation and Neromodulation of Hebei Province, Yanshan University, Qinhuangdao, Hebei 066004, P. R. China

E-mail Address: weiye51@ysu.edu.cn

Corresponding author.

Search for more papers by this author

, and

Wang Mingkui

College of Electrical Engineering, Yanshan University, Qinhuangdao, Hebei 066004, P. R. China

E-mail Address: wang_mk@yeah.net

Search for more papers by this author

https://doi.org/10.1142/S0219467823500274Cited by:0 (Source: Crossref)

Abstract

To solve the problem of road recognition when the robot is driving on the sidewalk, a novel sidewalk detection algorithm from the first-person perspective is proposed, which is crucial for robot navigation. The algorithm starts from the illumination invariance graph of the sidewalk image, and the sidewalk “seeds” are selected dynamically to get the sidewalk features for unsupervised feature learning. The final sidewalk region will be extracted by multi-threshold adaptive segmentation and connectivity processing. The key innovations of the proposed algorithm are the method of illumination invariance based on PCA and the unsupervised feature learning for sidewalk detection. With the PCA-based illumination invariance, it can calculate the lighting invariance angle dynamically to remove the impact of illumination and different brick colors’ influence on sidewalk detection. Then the sidewalk features are selected dynamically using the parallel geometric structure of the sidewalk, and the confidence region of the sidewalk is obtained through unsupervised feature learning. The proposed method can effectively suppress the effects of shadows and different colored bricks in the sidewalk area. The experimental result proves that the F-measure of the proposed algorithm can reach 93.11% and is at least 7.7% higher than the existing algorithm.

Keywords:

Remember to check out the Check out our Most Cited Articles!
Check out these titles on Image Analysis

References

1. K. Khalilullah, S. Ota, T. Yasuda and M. Jindai , “Road area detection method based on DBNN for robot navigation using single camera in outdoor environments,” Industrial Robot 45(2), 275–286 (2018). Crossref, Web of Science, Google Scholar
2. Z. J. Zhong and X. Qiang , “Research on the robot’s automatic road detection technology with SRPS,” in Proc. 2019 5th Int. Conf. Mechanical Engineering and Automation Science (ICMEAS) (2019), pp. 012001–012006. Google Scholar
3. Y. L. Zhu, D. Han, B. H. Xue et al., “Road curb detection using a novel tensor voting algorithm,” Int. Conf. Robotics and Biomimetics (ROBIO) (IEEE, 2019). Crossref, Google Scholar
4. H. Deng, P. Dong, Z. Li et al., “Robot navigation based on pseudo-binocular stereo vision and linear fitting,” 2020 IEEE Int. Conf. Integrated Circuits, Technologies and Applications (ICTA) (IEEE, 2020), pp. 174–175. Crossref, Google Scholar
5. R. Gregor, M. Lutzeler, M. Pellkofer et al., “EMS-vision: A perceptual system for autonomous vehicles,” IEEE Transactions on Intelligent Transportation Systems 3(1), 48–59 (2002). Crossref, Web of Science, Google Scholar
6. H. Loose and U. Franke , “B-spline-based road model for 3d lane recognition,” Int. IEEE Conf. Intelligent Transportation Systems (ITSC) (IEEE, 2010), pp. 91–98. Crossref, Google Scholar
7. M. Bertozzi, L. Bombini, A. Broggi et al., “GOLD: A framework for developing intelligent-vehicle vision applications,” IEEE Intelligent Systems 23(1), 69–71 (2008). Crossref, Web of Science, Google Scholar
8. G. Zhang, N. N. Zheng, C. Cui et al., “An efficient road detection method in noisy urban environment,” Intelligent Vehicles Symp. (IEEE, 2009), pp. 556–561. Crossref, Google Scholar
9. W. F. Wang, W. L. Ding, Y. Li et al., “An efficient road detection algorithm based on parallel edges,” Acta Optica Sinica 35(7), 224–230 (2015). Google Scholar
10. Z. G. Duan, Y. Li, E. D. Wang et al., “Road and navigation line detection algorithm from shadow image based on the illumination invariant image,” Acta Optica Sinica 36(12), 199–206 (2016). Google Scholar
11. S. Q. Fan, F. H. Zhu, H. Zhang et al., “Improving road detection results based on ensemble learning and key samples focusing,” 2020 IEEE 23rd Int. Conf. Intelligent Transportation Systems (ITSC) (IEEE, 2020), pp. 1–7. Crossref, Google Scholar
12. C. A. Brust, S. Sickert, M. Simon et al., “Convolutional patch networks with spatial prior for road detection and urban scene understanding,” Visapp 1, 82–90 (2015). Google Scholar
13. L. Qu, K. R. Wang, L. L. Chen et al., “Fast road detection based on RGBD images and convolutional neural network,” Acta Optica Sinica 37(10), 116–124 (2017). Google Scholar
14. J. M. Á. Alvarez and A. M. Lopez , “Road detection based on illuminant invariance,” IEEE Transactions on Intelligent Transportation Systems 12(1), 184–193 (2011). Crossref, Web of Science, Google Scholar
15. T. Almeida, B. Loureno and V. Santos , “Road detection based on simultaneous deep learning approaches,” Robotics and Autonomous Systems 133, 103605 (2020). Crossref, Web of Science, Google Scholar
16. X. Yang, X. Li, Y. Ye et al., “Road detection and centerline extraction via deep recurrent convolutional neural network U-Net,” IEEE Transactions on Geoscience and Remote Sensing 99, 1–12 (2019). Google Scholar
17. J. M. Alvarez, T. Gevers and A. M. Lopez, “Road detection by one-class color classification: Dataset and experiments,” arXiv:1412.3506, 2014. Google Scholar
18. C. C. T. Mendes, V. Frémont and D. F. Wolf , “Exploiting fully convolutional neural networks for fast road detection,” IEEE Int. Conf. Robotics and Automation (IEEE, 2016), pp. 3174–3179. Crossref, Google Scholar
19. L. Liu, Y. Wang and H. Zhao , “An image segmentation method for the blind sidewalks recognition by using the convolutional neural network U-Net,” 2019 IEEE Int. Conf. Signal, Information and Data Processing (ICSIDP) (IEEE, 2019), pp. 1–4. Crossref, Google Scholar
20. C. Matteo and S. Sergio , “Measuring urban sidewalk practicability: A sidewalk robot feasibility index,” IFAC-PapersOnLine 53(2), 15053–15058 (2020). Crossref, Google Scholar
21. Q. Hou and C. Ai , “A network-level sidewalk inventory method using mobile LiDAR and deep learning,” Transportation Research Part C: Emerging Technologies 119, 1–14 (2019). Web of Science, Google Scholar
22. J. Xu, X. C. Wang and Z. G. Fang , “Research and implementation of blind sidewalk detection in portable ETA system,” International Forum on Information Technology & Applications (IEEE Computer Society, 2010), pp. 431–434. Google Scholar
23. T. Wei and L. Yuan , “Highly real-time blind sidewalk recognition algorithm based on boundary tracking,” Opto-Electronic Engineering 44(7), 750 (2017). Google Scholar
24. M. Cheng, Y. Zhang, Y. Su et al., “Curb detection for road and sidewalk detection,” IEEE Transactions on Vehicular Technology 67(99), 10330–10342 (2018). Crossref, Web of Science, Google Scholar
25. D. Treccani, L. Díaz-Vilario and A. Adami , “Sidewalk detection and pavement characterisation in historic urban environments from point clouds: Preliminary results,” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII-B4-2021 (2021), pp. 243–249. Crossref, Google Scholar
26. Q. Hu, Q. Qu and X. He , “An improved sidewalk detection algorithm based on multi-feature fusion,” Applied Science and Technology 47(2), 35–43 (2020). Google Scholar
27. G. D. Finlayson, S. D. Hordley, C. Lu et al., “On the removal of shadows from images,” IEEE Transactions on Pattern Analysis & Machine Intelligence 28(1), 59–68 (2005). Crossref, Web of Science, Google Scholar
28. G. D. Finlayson, M. S. Drew and C. Lu , “Intrinsic images by entropy minimization,” European Conference on Computer Vision (ECCV) (2004), pp. 582–595. Google Scholar
29. T. Kim, Y. Tai and S. Yoon , “PCA based computation of illumination-invariant space for road detection,” IEEE Winter Conf. Applications of Computer Vision (WACV) (2017), pp. 632–640. Crossref, Google Scholar
30. D. W. Li, Wang W. Feng and L. X. Li , “OTLines: A novel line-detection algorithm without the interference of smooth curves,” Pattern Recognition 53, 238–258 (2016). Crossref, Web of Science, Google Scholar
31. B. Wang and V. Frémont , “Fast road detection from color images,” Intelligent Vehicles Symp. (IEEE, 2013), pp. 1209–1214. Crossref, Google Scholar
32. J. M. Álvarez, A. M. López, T. Gevers et al., “Combining priors, appearance, and context for road detection,” IEEE Transactions on Intelligent Transportation Systems 15(3), 1168–1178 (2014). Crossref, Web of Science, Google Scholar
33. J. K. Zhang, S. Y. Xia, K. Y. Lu et al., “Robust road detection from a single image,” 23rd Int. Conf. Pattern Recognition (ICPR) (2016), pp. 859–864. Crossref, Google Scholar
34. M. K. Wang, “Research on key technologies of obstacle detection in complex urban roads based on edge,” Yanshan University (2018). Google Scholar
35. C. Adam, Y. N. Andrew and L. Honglak , “An analysis of single-layer networks in unsupervised feature learning,” Int. Conf. Artificial Intelligence and Statistics (AISTATS) (2011), pp. 215–223. Google Scholar
36. A. Hyvärinen and E. Oja , “Independent component analysis: Algorithms and applications,” Neural Networks 13(4), 411–430 (2000). Crossref, Web of Science, Google Scholar
37. A. Ng , “Selecting receptive fields in deep networks,” Advances in Neural Information Processing Systems (2011), pp. 2528–2536. Google Scholar
38. J. L. Weng, “The research on class defect recognition method based on deep learning,” North University of China (2017), pp. 37–39. Google Scholar
39. L. He, Y. Chao, K. Suzuki et al., “Fast connected-component labeling,” Pattern Recognition 42(9), 1977–1987 (2009). Crossref, Web of Science, Google Scholar
40. Y. Li, W. L. Ding, X. G. Zhang et al., “Road detection algorithm for Autonomous Navigation Systems based on dark channel prior and vanishing point in complex road scenes,” Robotics & Autonomous Systems 85, 1–11 (2016). Crossref, Web of Science, Google Scholar