The accuracy of feature-based vision algorithms, including the self-calibration of binocular camera extrinsic parameters used in autonomous driving environment perception techniques relies heavily on the quality of the features extracted from the images. This study investigates the influence of the depth distance between objects and the camera, the feature points in different object regions, and the feature points in dynamic object regions on the self-calibration of binocular camera extrinsic parameters. To achieve this, the study first filters out different types of objects in the image through semantic segmentation. Then, it identifies the areas of dynamic objects and extracts the feature points in the static object region for the self-calibration of binocular camera extrinsic parameters. By calculating the baseline error of the binocular camera and the row alignment error of the matching feature points, this study evaluates the influence of feature points in dynamic object regions, feature points in different object regions, and feature points at different distances on the self-calibration algorithm. The experimental results demonstrate that feature points at static objects close to the camera are beneficial for the self-calibration of extrinsic parameters of binocular camera.

Keywords:

AMSC: 22E46, 53C35, 57S20

References

1. P. F. Alcantarilla, J. J. Yebes, J. Almazán and L. M. Bergasa , On combining visual slam and dense scene flow to increase the robustness of localization and mapping in dynamic environments, in IEEE Int. Conf. Robotics and Automation (RiverCentre, Saint Paul, Minnesota, USA, 2012), pp. 1290–1297. Crossref, Google Scholar
2. A. C. Berg, T. L. Berg and J. Malik , Shape matching and object recognition using low distortion correspondences, IEEE Comput. Soc. Conf. Computer Vision and Pattern Recognition (San Diego, CA, USA, 2005), pp. 26–33. Crossref, Google Scholar
3. M. V. Berry, Z. Lewis and J. F. Nye , On the weierstrass-mandelbrot fractal function, Proc. R. Soc. London A Math. Phys. Sci. 370(1743) (1980) 459–484. Crossref, Web of Science, Google Scholar
4. V. Bolon-Canedo and B. Remeseiro , Feature selection in image analysis: A survey, Artif. Intell. Rev. 53(4) (2020) 2905–2931. Crossref, Web of Science, Google Scholar
5. J.-Y. Bouguet , Visual Methods for Three Dimensional Modeling (California Institute of Technology, 1999). Google Scholar
6. S. Chen, N. Chen, Z. Sun and R. Meng , Self-calibration and optimization of binocular camera extrinsic parameters, in Int. Conf. Digital Twins and Parallel Intelligence (Beijing, China, 2021), pp. 139–143. Crossref, Google Scholar
7. T. Chen, H. Yuan and M. Yin , Hierarchical unsupervised multi-view feature selection, Int. J. Wavelets, Multiresolution Inf. Process. 20(6) (2022) 2250024. Link, Web of Science, Google Scholar
8. A. M. G. de Almeida, C. H. Recco and R. C. Guido , Use of paraconsistent feature engineering to support the long term feature choice for speaker verification, Proc. FL AIRS 34 (2021). Google Scholar
9. N. Dhanachandra, K. Manglem and Y. J. J. P. C. S. Chanu , Image segmentation using k-means clustering algorithm and subtractive clustering algorithm, Proc. Comput. Sci. 54 (2015) 764–771. Crossref, Google Scholar
10. A. Fusiello, E. Trucco and A. Verri , A compact algorithm for rectification of stereo pairs, Mach. Vision Appl. 12(1) (2000) 16–22. Crossref, Web of Science, Google Scholar
11. S. Gidaris and N. Komodakis , Detect, replace, refine: Deep structured prediction for pixel wise labeling, in Proc. IEEE Conf. Computer Vision and Pattern Recognition (2017), pp. 5248–5257. Crossref, Google Scholar
12. E. Guariglia , Harmonic sierpinski gasket and applications, Entropy 20(9) (2018) 714. Crossref, Web of Science, Google Scholar
13. E. Guariglia , Primality, fractality, and image analysis, Entropy 21(3) (2019) 304. Crossref, Web of Science, Google Scholar
14. E. Guariglia and R. C. Guido , Chebyshev wavelet analysis, J. Funct. Spaces (2022) 5542054. Google Scholar
15. E. Guariglia and S. Silvestrov , Fractional-wavelet analysis of positive definite distributions and wavelets on $d^{'} (c)$ , Eng. Math. II 179 (2016) 337–353. Google Scholar
16. R. C. Guido, F. Pedroso, R. C. Contreras, L. C. Rodrigues, E. Guariglia and J. S. Neto , Introducing the discrete path transform (dpt) and its applications in signal analysis, artefact removal, and spoken word recognition, Digital Signal Process. 117 (2021) 103158. Crossref, Web of Science, Google Scholar
17. R. Hartley and A. Zisserman , Multiple View Geometry in Computer Vision (Cambridge University Press, 2003). Google Scholar
18. N. Karlsson, E. D. Bernardo, J. Ostrowski, L. Goncalves, P. Pirjanian and M. E. Munich , The vslam algorithm for robust localization and mapping, in Proc. IEEE Int. Conf. Robotics and Automation (Barcelona, Spain, 2005), pp. 24–29. Crossref, Google Scholar
19. A. Kendall and R. Cipolla , Geometric loss functions for camera pose regression with deep learning, in Proc. IEEE Conf. Computer Vision and Pattern Recognition (Honolulu, HI, USA, 2017), pp. 5974–5983. Crossref, Google Scholar
20. X. Li, B. Ge, Q. Luo, Y. Li and Q. Tian , Acquisition of camera dynamic extrinsic parameters in free binocular stereo vision system, J. Comput. Appl. 37(10) (2017) 2888. Google Scholar
21. Q. Long, Q. Xie, S. Mita, K. Ishimaru and N. Shirai , A real-time dense stereo matching method for critical environment sensing in autonomous driving, in Int. IEEE Conf. Intelligent Transportation Systems (Qingdao, China, 2014), pp. 853–860. Google Scholar
22. Q. Long, Q. Xie, S. Mita, H. T. Niknejad, K. Ishimaru and C. Guo , Real-time dense disparity estimation based on multi-path viterbi for intelligent vehicle applications, in Proc. British Machine Vision Conference (Nottingham, UK, 2014), pp. 1–13. Crossref, Google Scholar
23. D. G. J. I. J. o. C. V. Lowe , Distinctive image features from scale-invariant keypoints, Int. J. Comput. Vision 60(2) (2004) 91–110. Crossref, Web of Science, Google Scholar
24. C. Ma, S. Pei, G. Sun, R. Meng and K. Luo , Disparity estimation based on fusion of vision and lidar, Int. J. Wavelets Multiresolution Inf. Process. 20(5) (2022) 2250014. Link, Web of Science, Google Scholar
25. M. Miksch, B. Yang and K. Zimmermann , Automatic extrinsic camera self-calibration based on homography and epipolar geometry, IEEE Intell. Vehicles Symp. (La Jolla, CA, USA, 2010), pp. 832–839. Crossref, Google Scholar
26. M. Muja and D. G. Lowe , Fast approximate nearest neighbors with automatic algorithm configuration, VISAPP 2(331–340) (2009) 2. Google Scholar
27. D. Nister , An efficient solution to the five-point relative pose problem, IEEE Comput. Soc. Conf. Comput. Vision Pattern Recogn. 2 (2003) 22–195. Google Scholar
28. N. Parmar, A. Vaswani, J. Uszkoreit, L. Kaiser, N. Shazeer, A. Ku and D. Tran , Image transformer, Int. Conf. Mach. Learning (Stockholm, Sweden, 2018) 4055–4064. Google Scholar
29. F. I. Pereira, J. A. Luft, G. Ilha and A. J. I. T. O. I. T. S. Susin , A novel resection-intersection algorithm with fast triangulation applied to monocular visual odometry, IEEE Trans. Intell. Transp. Syst. 19(11) (2018) 3584–3593. Crossref, Web of Science, Google Scholar
30. P. H. Torr and A. Zisserman , Feature based methods for structure and motion estimation, in Int. Workshop Vision Algorithms (Corfu, Greece, 1999), pp. 278–294. Google Scholar
31. J. Vargas, S. Alsweiss, O. Toker, R. Razdan and J. Santos , An overview of autonomous vehicles sensors and their vulnerability to weather conditions, Sensors 21(16) (2021) 5397. Crossref, Web of Science, Google Scholar
32. M. E. Wall, A. Rechtsteiner and L. M. Rocha , Singular value decomposition and principal component analysis, A Practical Approach to Microarray Data Analysis (Springer, Boston, 2003), pp. 91–109. Crossref, Google Scholar
33. Wang, Chiao, Sawchuk and A. Alexander , Disparity manipulation for stereo images and video, Stereoscopic Displays Appl. 19 (2008) 473–484. Google Scholar
34. P. Wang, P. Chen, Y. Yuan, D. Liu, Z. Huang, X. Hou and G. Cottrell , Understanding convolution for semantic segmentation, IEEE Winter Conf. Applications of Computer Vision (Lake Tahoe, NV, USA, 2018), pp. 1451–1460. Crossref, Google Scholar
35. R. Wang, W. Wan, Y. Wang and K. J. R. S. Di , A new RGB-D slam method with moving object detection for dynamic indoor scenes, Remote Sensing 11(10) (2019) 1143. Crossref, Web of Science, Google Scholar
36. Y. Wang, W. Huang, Z. Xu and M. Wang , Adaptive compensation visual odometry in dynamic scenarios, Int. J. Wavelets, Multiresolution Inf. Process. 20(4) (2022) 2250003. Link, Web of Science, Google Scholar
37. Y. Wei, J. Feng, X. Liang, M. M. Cheng, Y. Zhao and S. Yan , Object region mining with adversarial erasing: A simple classification to semantic segmentation approach, in IEEE Conf. Comput. Vision Pattern Recognition (Honolulu, HI, USA, 2017), pp. 1568–1576. Crossref, Google Scholar
38. Q. Xie, X. Chen, L. Zhang, A. Jiang and F. Cui , A robust and efficient video anti-shaking algorithm for low-end smartphone platforms, IEEE Trans. Consumer Electron. (2019) 1–10. Crossref, Web of Science, Google Scholar
39. Q. Xie, X. Hu, L. Ren, L. Qi and Z. Sun , A binocular vision application in iot: Realtime trustworthy road condition detection system in passable area, IEEE Trans. Ind. Inf. 19(1) (2023) 973–983. Crossref, Web of Science, Google Scholar
40. Q. Xie, R. Liu, Z. Sun, S. Pei and F. Cui , A flexible free-space detection system based on stereo vision, Neurocomputing 485 (2022) 252–262. Crossref, Web of Science, Google Scholar
41. Q. Xie, Q. Long and S. Mita , Integration of optical flow and multi-path-viterbi algorithm for stereo vision, Int. J. Wavelets, Multiresolution Inf. Process. 15(3) (2017) 1750022. Link, Web of Science, Google Scholar
42. D. Xu, D. Zhang, X. Liu, L. J. I. T. O. I. Ma and Measurement , A calibration and 3-d measurement method for an active vision system with symmetric yawing cameras, IEEE Trans. Instrum. Measurement 70 (2021) 1–13. Crossref, Web of Science, Google Scholar
43. L. Yang, H. Su, C. Zhong, Z. Meng and Y. Lu , Hyperspectral image classification using wavelet transform-based smooth ordering, Int. J. Wavelets Multiresolution Inf. Process. 17(10) (2019) 1950050. Link, Web of Science, Google Scholar
44. H. Yin, Z. Ma, M. Zhong, K. Wu, Y. Wei, J. Guo and B. J. S. Huang , Slam-based self-calibration of a binocular stereo vision rig in real-time, Sensors 20(3) (2020) 621. Crossref, Web of Science, Google Scholar
45. Y. Yuan, X. Chen and J. Wang , Object-contextual representations for semantic segmentation, in European Conf. Computer Vision (Tel Aviv, Israel, 2022), pp. 173–190. Google Scholar
46. Y. Yuan, L. Huang, J. Guo, C. Zhang, X. Chen and J. Wang, Ocnet: Object context network for scene parsing, preprint (2018), arXiv:1809.00916, pp. 1–22. Google Scholar
47. Y. Yuan, J. Xie, X. Chen and J. Wang , Segfix: Model-agnostic boundary refinement for segmentation, in European Conf. Computer Vision (Glasgow, UK, 2020), pp. 489–506. Crossref, Google Scholar
48. H. Zhang, K. Dana, J. Shi, Z. Zhang, X. Wang, A. Tyagi and A. Agrawal , Context encoding for semantic segmentation, in Proc. IEEE Conf. Computer Vision and Pattern Recognition (Salt Lake City, UT, USA, 2018), pp. 7151–7160. Crossref, Google Scholar
49. L. Zhang, X. Li, A. Arnab, K. Yang, Y. Tong and P. Torr , Dual graph convolutional network for semantic segmentation, in Proc. British Machine Vision Conference (Cardiff, UK, 2019), pp. 1–18. Google Scholar
50. Z. Zhang , A flexible new technique for camera calibration, IEEE Trans. Pattern Anal. Mach. Intelligence 22(11) (2000) 1330–1334. Crossref, Web of Science, Google Scholar
51. S. Zhao, M. Chao, W. Liang, M. Ran and P. Shanshan , A deep learning-based binocular perception system, J. Syst. Eng. Electron. 32(1) (2021) 7–20. Crossref, Web of Science, Google Scholar
52. X. Zheng, Y. Y. Tang and J. Zhou , A framework of adaptive multiscale wavelet decomposition for signals on undirected graphs, IEEE Trans. Signal Process. 67(7) (2019) 1696–1711. Crossref, Web of Science, Google Scholar