Each person describes unique patterns during gait cycles and this information can be extracted from live video stream and used for subject identification. In recent years, there has been a profusion of sensors that in addition to RGB video images also provide depth data in real-time. In this paper, a method to enhance the appearance-based gait recognition method by also integrating features extracted from depth data is proposed. Two approaches are proposed that integrate simple depth features in a way suitable for real-time processing. Unlike previously presented works which usually use a short range sensors like Microsoft Kinect, here, a long-range stereo camera in outdoor environment is used. The experimental results for the proposed approaches show that recognition rates are improved when compared to existing popular gait recognition methods.

This paper was recommended by Regional Editor Zoran Stamenkovic.

Keywords:

References

1. J. Han and B. Bhanu, Individual recognition using gait energy image, IEEE Trans. Pattern Anal. Mach. Intell. 28 (2006) 316–322. Crossref, Web of Science, Google Scholar
2. I. B. Barbosa, M. Cristani, A. Del Bue, L. Bazzani and V. Murino, Re-identification with rgb-d sensors, in 12th European Conf. Computer Vision (ECCV 2012) (Springer, Berlin Heidelberg, Florance, Italy, 2012), pp. 433–442. Crossref, Google Scholar
3. W. Choi, C. Pantofaru and S. Savarese, Detecting and tracking people using an rgb-d camera via multiple detector fusion, in IEEE Int. Conf. Computer Vision Workshops (ICCV Workshops 2011) (Barcelona, Spain, 2011), pp. 1076–1083. Crossref, Google Scholar
4. M. Munaro and E. Menegatti, Fast rgb-d people tracking for service robots, Auton. Robots 37 (2014) 227–242. Crossref, Web of Science, Google Scholar
5. A. Møgelmose, C. Bahnsen, T. B. Moeslund, A. Clapes and S. Escalera, Tri-modal person re-identification with rgb, depth and thermal features, in IEEE Conf. Computer Vision and Pattern Recognition Workshops (CVPR 2013) (Portland, Oregon, 2013), pp. 301–307. Crossref, Google Scholar
6. R. Satta, F. Pala, G. Fumera and F. Roli, Real-time appearance-based person re-identification over multiple kinect cameras, in 8th Int. Conf. Computer Vision Theory and Applications (VISAPP 2013) (Barcelona, Spain, 2013), pp. 407–410. Google Scholar
7. J. Lorenzo-Navarro, M. Castrillón-Santana and D. Hernández-Sosa, On the use of simple geometric descriptors provided by rgb-d sensors for re-identification, Sensors 13 (2013) 8222–8238. Crossref, Web of Science, Google Scholar
8. M. Munaro, A. Fossati, A. Basso, E. Menegatti and L. Van Gool, One-shot person re-identification with a consumer depth camera, Adv. Comput. Vis. Pattern Recogn. (2014) 161–181. Crossref, Google Scholar
9. A. Møgelmose, T. B. Moeslund and K. Nasrollahi, Multimodal person re-identification using rgb-d sensors and a transient identification database, in Int. Workshop on Biometrics and Forensics (IWBF 2013) (Lisabon, Portugal, 2013), pp. 1–4. Crossref, Google Scholar
10. M. Munaro, S. Ghidoni, D. T. Dizmen and E. Menegatti, A feature-based approach to people re-identification using skeleton keypoints, in IEEE Int. Conf. Robotics and Automation (ICRA 2014) (Hong Kong, China, 2014), pp. 5644–5651. Crossref, Google Scholar
11. A. Ramakić and A. Toroman, People identification using kinect sensor, in CMBEBIH 2017 (Springer, 2017) pp. 277–282. Crossref, Google Scholar
12. S. Sivapalan, D. Chen, S. Denman, S. Sridharan and C. Fookes, Gait energy volumes and frontal gait recognition using depth images, in Int. Joint Conf. Biometrics (IJCB 2011) (Washington DC, 2011), pp. 1–6. Crossref, Google Scholar
13. M. Hofmann, S. Bachmann and G. Rigoll, 2.5d gait biometrics using the depth gradient histogram energy image, in IEEE Fifth Int. Conf. Biometrics: Theory, Applications and Systems (BTAS 2012) (Arlington, Virginia, 2012), pp. 399–403. Crossref, Google Scholar
14. R. Borràs, À. Lapedriza and L. Igual, Depth information in human gait analysis: An experimental study on gender recognition, in 9th Int. Conf. Image Analysis and Recognition (ICIAR 2012) (Springer, Berlin Heidelberg, Aveiro, Portugal, 2012), pp. 98–105. Crossref, Google Scholar
15. J. Lu, G. Wang and P. Moulin, Human identity and gender recognition from gait sequences with arbitrary walking directions, IEEE Trans. Inf. Forensics Security 9 (2014) 51–61. Crossref, Web of Science, Google Scholar
16. P. Chattopadhyay, A. Roy, S. Sural and J. Mukhopadhyay, Pose depth volume extraction from rgb-d streams for frontal gait recognition, J. Vis. Commun. Image Rep. 25 (2014) 53–63. Crossref, Web of Science, Google Scholar
17. P. Arora and S. Srivastava, Gait recognition using gait gaussian image, in 2nd Int. Conf. Signal Processing and Integrated Networks (SPIN 2015) (Noida, India, 2015), pp. 791–794. Crossref, Google Scholar
18. A. M. Nambiar, P. L. Correia and L. D. Soares, Frontal gait recognition combining 2d and 3d data, in Proc. Multimedia and Security (MMSEC 2012) (ACM, Coventry, United Kingdom, 2012), pp. 145–150. Crossref, Google Scholar
19. Y. Iwashita, K. Uchino and R. Kurazume, Gait-based person identification robust to changes in appearance, Sensors 13 (2013) 7884–7901. Crossref, Web of Science, Google Scholar
20. S. Sivapalan, D. Chen, S. Denman, S. Sridharan and C. Fookes, The backfilled gei - a cross-capture modality gait feature for frontal and side-view gait recognition, in Int. Conf. Digital Image Computing Techniques and Applications (DICTA 2012) (Fremantle, Australia, 2012), pp. 1–8. Crossref, Google Scholar
21. J. Preis, M. Kessel, M. Werner and C. Linnhoff-Popien, Gait recognition with kinect, in 1st Int. Workshop on Kinect in Pervasive Computing (New Castle, UK, 2012), pp. 1–4. Google Scholar
22. M. S. N. Kumar and R. V. Babu, Human gait recognition using depth camera: A covariance based approach, in Proc. 8th Indian Conf. Computer Vision, Graphics and Image Processing (ICVGIP 2012) (ACM, Mumbai, India, 2012), pp. 20:1–20:6. Crossref, Google Scholar
23. A. Sinha, K. Chakravarty and B. Bhowmick, Person identification using skeleton information from kinect, in Proc. 6th Int. Conf. Advances in Computer – Human Interactions (ACHI 2013) (Nice, France, 2013), pp. 101–108. Google Scholar
24. A. Ball, D. Rye, F. Ramos and M. Velonaki, Unsupervised clustering of people from ‘skeleton’ data, in 7th ACM/IEEE Int. Conf. Human-Robot Interaction (HRI 2012) (Boston, Massachusetts, 2012), pp. 225–226. Crossref, Google Scholar
25. J. Kovač and P. Peer, Human skeleton model based dynamic features for walking speed invariant gait recognition, Math. Problems Eng. 2014 (2014) 15. Crossref, Web of Science, Google Scholar
26. K. Arai and R. A. Asmara, 3d skeleton model derived from kinect depth sensor camera and its application to walking style quality evaluations, Int. J. Adv. Res. Artificial Intell. 2 (2013) 24–28. Crossref, Google Scholar
27. F. M. Castro, M. J. Marín-Jiménez, N. Guil and N. P. de la Blanca, Automatic learning of gait signatures for people identification, in Int. Work-Conf. Artificial Neural Networks, Springer, 2017, pp. 257–270. Crossref, Google Scholar
28. M. Alotaibi and A. Mahmood, Improved gait recognition based on specialized deep convolutional neural network, Comput. Vis. Image Understand. 164 (2017) 103–110. Crossref, Web of Science, Google Scholar
29. Q. Zou, L. Ni, Q. Wang, Q. Li and S. Wang, Robust gait recognition by integrating inertial and rgbd sensors, IEEE Trans. Cybern. 48 (2018) 1136–1150. Crossref, Web of Science, Google Scholar
30. A. O. Lishani, L. Boubchir, E. Khalifa and A. Bouridane, Human gait recognition based on haralick features, Signal, Image Video Process. 11 (2017) 1123–1130. Crossref, Web of Science, Google Scholar
31. W. Li, C.-C. J. Kuo and J. Peng, Gait recognition via gei subspace projections and collaborative representation classification, Neurocomputing 275 (2018) 1932–1945. Crossref, Web of Science, Google Scholar
32. Y. Wang, Y. Chen, M. Z. A. Bhuiyan, Y. Han, S. Zhao and J. Li, Gait-based human identification using acoustic sensor and deep neural network, Future Gen. Comput. Syst. 86 (2018) 1228–1237. Crossref, Web of Science, Google Scholar
33. J. Portillo-Portillo, R. Leyva, V. Sanchez, G. Sanchez-Perez, H. Perez-Meana, J. Olivares-Mercado, K. Toscano-Medina and M. Nakano-Miyatake, A view-invariant gait recognition algorithm based on a joint-direct linear discriminant analysis, Appl. Intell. 48 (2018) 1200–1217. Web of Science, Google Scholar
34. S. Zheng, J. Zhang, K. Huang, R. He and T. Tan, Robust view transformation model for gait recognition, in 18th IEEE Int. Conf. Image Process. (ICIP 2011), Sep 2011, pp. 2073–2076. Google Scholar
35. S. Yu, D. Tan and T. Tan, A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition, in 18th Int. Conf. Pattern Recognition (ICPR 06), 2006, pp. 441–444. Google Scholar
36. Center for Biometrics and Security Research, Institute of Automation Chinese Academy of Sciences, http://www.cbsr.ia.ac.cn/english/Gait%20Databases.asp. Google Scholar
37. M. A. Fischler and R. C. Bolles, Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography, Commun. ACM 24 (1981) 381–395. Crossref, Web of Science, Google Scholar
38. R. Kurban, F. Skuka and H. Bozpolat, Plane segmentation of kinect point clouds using RANSAC, The 7th Int. Conf. Information Technology (2015), pp. 545–551. Crossref, Google Scholar
39. J. Biswas and M. Veloso, Planar polygon extraction and merging from depth images, in IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS 2012) (IEEE, 2012), pp. 3859–3864. Crossref, Google Scholar
40. H. Bay, A. Ess, T. Tuytelaars and L. Van Gool, Speeded-up robust features (surf), Comput. Vis. Image Underst. 110 (2008) 346–359. Crossref, Web of Science, Google Scholar
41. G. Csurka, C. R. Dance, L. Fan, J. Willamowski and C. Bray, Visual categorization with bags of keypoints, in In Workshop on Statistical Learning in Computer Vision (ECCV 2004), 2004, pp. 1–22. Google Scholar
42. Mathworks, Statistics and Machine Learning, https://www.mathworks.com/discovery/cross-validation.html. Google Scholar
43. A. Ross and A. Jain, Information fusion in biometrics, Pattern Recogn. Lett. 24 (2003) 2115–2125. Crossref, Web of Science, Google Scholar