No Access

Gait Recognition Based on Minirocket with Inertial Measurement Units

Institute of Intelligent Perception and Diagnosis, School of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237, P. R. China

E-mail Address: jjyan@ecust.edu.cn

Corresponding author.

Search for more papers by this author

Weixiang Xiong

https://orcid.org/0000-0002-6874-4765

Institute of Intelligent Perception and Diagnosis, School of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237, P. R. China

E-mail Address: 2905430944@qq.com

Search for more papers by this author

Li Jin

Servo Department, Shanghai Aerospace Control Technology Research Institute, Shanghai 201108, P. R. China

E-mail Address: jinli9617@163.com

Search for more papers by this author

Jinlin Jiang

Servo Department, Shanghai Aerospace Control Technology Research Institute, Shanghai 201108, P. R. China

E-mail Address: jiang-jjl@163.com

Search for more papers by this author

Zhihao Yang

Institute of Intelligent Perception and Diagnosis, School of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237, P. R. China

E-mail Address: 1040126414@qq.com

Search for more papers by this author

Jiangtao Shi

Institute of Intelligent Perception and Diagnosis, School of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237, P. R. China

E-mail Address: 1399629096@qq.com

Search for more papers by this author

, and

Shuai Hu

Institute of Intelligent Perception and Diagnosis, School of Mechanical and Power Engineering, East China University of Science and Technology, Shanghai 200237, P. R. China

E-mail Address: 1964840084@qq.com

Search for more papers by this author

https://doi.org/10.1142/S0219843623500093Cited by:2 (Source: Crossref)

Abstract

Gait recognition is one of the key technologies for exoskeleton robot control. The recognition accuracy and robustness of existing gait recognition methods cannot well meet the needs of real-time control. There is still room for improvement in fine-grained gait recognition. In this regard, this paper proposes a gait recognition method based on the MiniRocket and inertial measurement units. In this paper, a human lower limb posture information collection device is developed to collect ten kinds of gait data of human lower limbs (walking, standing, running, backing off, going upstairs, going downstairs, going uphill, going downhill, stand at ease and squat). The MiniRocket algorithm was used to build a human gait recognition model, and the effects of algorithm parameters and the size of the window and shift on the performance of gait recognition were discussed, and user-independent experiments and user-dependent experiments were carried out, respectively, and compared with four algorithms of TST, TCN, RNN and LSTM. The experimental results show that the MiniRocket algorithm has an average recognition accuracy of 94.87% and 97.67% in the user-independent experiment and the user-dependent experiment, which is better than the other four algorithms. And the effectiveness of the method in the IMUs-based human gait recognition problem is shown, which provides some implications for fine-grained gait recognition.

Keywords:

References

1. D. P. Ferrris, G. S. Sawicki and M. A. Daley, A physiologist’s perspective on robotic exoskeletons for human locomotion, Int. J. Humanoid Robot. 4(3) (2007) 507–528. Link, Web of Science, Google Scholar
2. C. Walsh et al., Development of a lightweight, underactuated exoskeleton for load-carrying augmentation, in IEEE Int. Conf. Robotics and Automation (IEEE, 2006), pp. 3485–3491. Crossref, Google Scholar
3. C. J. Walsh, K. Endo and H. Herr, A quasi-passive leg exoskeleton for load-carrying augmentation, Int. J. Humanoid Robot. 4(3) (2007) 487–506. Link, Web of Science, Google Scholar
4. LG to usher in new era of AI robotics with wearable robot [EB/OL] (2018), http://www. lgnewsroom.com/2018/08/lg-to-usher-in-new-era-of-ai-robotics-with-wearable-robot/. Google Scholar
5. Lockheed Martin, Lockheed Martin partners with Deakin University to further develop industrial exoskeleton [EB/OL] (2018), https://news.lockheedmartin.com. Google Scholar
6. K. Strausser and H. Kazerooni, The development and testing of a human machine interface for a mobile medical exoskeleton, in IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IEEE, 2011), pp. 4911–4916. Crossref, Google Scholar
7. S. Wang et al., Design and control of the MINDWALKER exoskeleton, IEEE Trans. Neural Syst. Rehabil. Eng. 23(2) (2015) 277–286. Crossref, Web of Science, Google Scholar
8. G. Cheng et al., BPNN-based real-time recognition of locomotion modes for an active pelvis orthosis with different assistive strategies, Int. J. Humanoid Robot. 17(1) (2020) 2050004. Link, Web of Science, Google Scholar
9. J. Li, Research and implementation of the exoskeleton gait detection system, School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu (2013), pp. 1–3. Google Scholar
10. J. Sun et al., Research on sensing systems of wearable power assist leg, Micronanoelectron. Technol. 44(7) (2007) 353–357. Google Scholar
11. T. Hayashi, H. Kawamoto and Y. Sankai, Control method of robot suit HAL working as operator’s muscle using biological and dynamical information, in 2005 IEEE/RSJ Int. Conf. Intelligent Robots and Systems, Edmonton, Alberta (IEEE, 2005), pp. 2–6. Google Scholar
12. H. Kawamoto and Y. Sankai, Power assist method based on phase sequence and muscle force condition for HAL, Adv. Robot. 19(7) (2005) 717–734. Crossref, Web of Science, Google Scholar
13. K. Suzuki et al., Intention-based walking support for paraplegia patients with robot suit HAL, Adv. Robot. 21(12) (2007) 1441–1469. Crossref, Web of Science, Google Scholar
14. A. Tsukahara et al., Sit-to-stand and stand-to-sit transfer support for complete paraplegic patients with robot suit HAL, Adv. Robot. 24(11) (2010) 1615–1638. Crossref, Web of Science, Google Scholar
15. J. L. Contrerasvidal and R. G. Grossman, NeuroRex: A clinical neural interface roadmap for EEG-based brain machine interfaces to a lower body robotic exoskeleton, in Int. Conf. IEEE Engineering in Medicine and Biology Society (IEEE, 2013), pp. 1579–1582. Google Scholar
16. A. Kilicarslan et al., High accuracy decoding of user intentions using EEG to control a lower-body exoskeleton, in Int. Conf. IEEE Engineering in Medicine and Biology Society (IEEE, 2013), pp. 5606–5609. Crossref, Google Scholar
17. R. W. Nuckols et al., Individualization of exosuit assistance based on measured muscle dynamics during versatile walking, Sci. Robot. 6 (2021) eabj1362. Crossref, Web of Science, Google Scholar
18. J. Mantyjarvi et al., Identifying users of portable devices from gait pattern with accelerometers, in IEEE Int. Conf. Acoustics (IEEE, 2005), pp. 973–976. Crossref, Google Scholar
19. A. J. Young, A. M. Simon and L. J. Hargrove, A training method for locomotion modeprediction using powered lower limb prostheses, IEEE Trans. Neural Syst. Rehabil. Eng. 22(3) (2013) 671–677. Crossref, Web of Science, Google Scholar
20. F. L. Haufe et al., User-driven walking assistance: First experimental results using the MyoSuit, in 2019 IEEE 16th Int. Conf. Rehabilitation Robotics (ICORR) (IEEE, 2019), pp. 944–949. Crossref, Google Scholar
21. H. Kazerooni, R. Steger and L. Huang, Hybrid control of the Berkeley lower extremity exoskeleton (BLEEX), Int. J. Robot. Res. 25(25) (2006) 561–573. Crossref, Web of Science, Google Scholar
22. H. Kazerooni, The Berkeley lower extremity exoskeleton, J. Dynam. Syst. Meas. Control 128(1) (2006) 9–15. Crossref, Web of Science, Google Scholar
23. F. Massé et al., Improving activity recognition using a wearable barometric pressure sensor in mobility-impaired stroke patients, J. Neuroeng. Rehabil. 12(1) (2015) 72–82. Crossref, Web of Science, Google Scholar
24. L. Lei et al., Human motion intent recognition based on kernel principal component analysis and relevance vector machine, Robot 39(5) (2017) 661–669. Google Scholar
25. J. H. Dong, H. Lim and K. Jung, Development of ankle-less active lower-limb exoskeleton controlled using finite leg function state machine, Int. J. Precis. Eng. Manuf. 18(6) (2017) 803–811. Crossref, Web of Science, Google Scholar
26. H. U. Martinez and S. A. Dehghani, Adaptive Bayesian inference system for recognition of walking activities and prediction of gait events using wearable sensors, Neural Netw. 102(1) (2018) 107–119. Crossref, Web of Science, Google Scholar
27. Y. L. Hsu et al., Human daily and sport activity recognition using a wearable inertial sensor network, IEEE Access 6 (2018) 31715–31728. Crossref, Web of Science, Google Scholar
28. J. A. Young et al., A training method for locomotion mode prediction using powered lower limb prostheses, Neural Syst. Rehabil. Eng. 22(3) (2014) 671–677. Crossref, Web of Science, Google Scholar
29. S. M. Lee, M. Y. Sang and H. Cho, Human activity recognition from accelerometer data using Convolutional Neural Network, in IEEE Int. Conf. Big Data & Smart Computing (IEEE, 2017), pp. 131–134. Google Scholar
30. R. Liao et al., Pose-based temporal spatial network (PTSN) for gait recognition with carrying and clothing variations, in Proc. Chinese Conf. Biometric Recognition (Springer, Berlin, 2017), pp. 474–483. Crossref, Google Scholar
31. S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Comput. 9(8) (1997) 1735–1780. Crossref, Web of Science, Google Scholar
32. F. Yang, Y. Li and J. Luo, Learning effective gait features using LSTM, in Proc. 23rd IEEE Int. Conf. Pattern Recognition (IEEE, 2017), pp. 325–330. Google Scholar
33. T. Pfister, J. Charles and A. Zisserman, Flowing ConvNets for human pose estimation in videos, in Proc. IEEE Int. Conf. Computer Vision (IEEE, 2016), pp. 1913–1921. Google Scholar
34. S. Yu et al., GaitGAN: Invariant gait feature extraction using generative adversarial networks, in Proc. IEEE Conf. Computer Vision and Pattern Recognition (IEEE, 2017), pp. 30–37. Crossref, Google Scholar
35. M. Mirza and S. Osindero, Conditional generative adversarial nets, Comput. Sci. (2014), pp. 2672–2680. Google Scholar
36. X. Chen et al., InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets, in Proc. 30th Int. Conf. Neural Information Processing Systems (NIPS, 2016), pp. 2180–2188. Google Scholar
37. B. Fang et al., Gait neural network for human-exoskeleton interaction, Front. Neurorobot. 14 (2020) 1–9. Crossref, Web of Science, Google Scholar
38. Z. Wu et al., A comprehensive study on cross-view gait based human identification with deep CNNs, IEEE Trans. Pattern Anal. Mach. Intell. 39(2) (2017) 209–226. Crossref, Web of Science, Google Scholar
39. X. Zhang et al., DeepGait: A learning deep convolutional representation for gait recognition, in Proc. Chinese Conf. Biometric Recognition (Springer, Berlin, 2017), pp. 447–456. Crossref, Google Scholar
40. K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, Comput. Sci. (2014). Web of Science, Google Scholar
41. A. Dempster, D. F. Schmidt and G. I. Webb, MiniRocket: A very fast (almost) deterministic transform for time series classification, in KDD’21: The 27th ACM SIGKDD Conf. Knowledge Discovery and Data Mining (ACM, 2021), pp. 248–257. Crossref, Google Scholar
42. Y. Wang, Gait modeling and experimental study of the human–machine system of the lower extremity assisted exoskeleton, East China University of Science and Technology, Shanghai (2021). Google Scholar
43. A. Dempster, F. Petitjean and G. I. Webb, ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels, Data Min. Knowl. Discov. 34(5) (2020) 1454–1495. Crossref, Web of Science, Google Scholar
44. S. Li et al., Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting (2019). Google Scholar
45. B. Shaojie, J. Zico Kolter and V. Koltun, An empirical evaluation of generic convolutional and recurrent networks for sequence modeling (2018). Google Scholar
46. J. L. Elman, Finding structure in time, Cogn. Sci. 14(2) (1990) 179–211. Crossref, Web of Science, Google Scholar
47. Q. Ai et al., A subject-specific EMG-driven musculoskeletal model for applications in lower-limb rehabilitation robotics, Int. J. Humanoid Robot. 13 (2016) 1650005. Link, Web of Science, Google Scholar
48. D. Torricelli et al., A subject-specific kinematic model to predict human motion in exoskeleton-assisted gait, Front. Neurorobot. 12 (2018) 1–11. Crossref, Web of Science, Google Scholar