Research PaperNo Access

Student body gesture recognition based on Fisher broad learning system

Yafei Shi

School of Educational Information Technology, Central China Normal University, Wuhan 430079, P. R. China

E-mail Address: yafeishi@mails.ccnu.edu.cn

Search for more papers by this author

Yantao Wei

http://orcid.org/0000-0001-7225-1958

School of Educational Information Technology, Central China Normal University, Wuhan 430079, P. R. China

E-mail Address: yantaowei@mail.ccnu.edu.cn

Corresponding author.

Search for more papers by this author

Donghui Pan

School of Mathematical Sciences, Anhui University, Hefei 230601, P. R. China

E-mail Address: dhpan@ahu.edu.cn

Search for more papers by this author

Wei Deng

School of Educational Information Technology, Central China Normal University, Wuhan 430079, P. R. China

Search for more papers by this author

Huang Yao

School of Educational Information Technology, Central China Normal University, Wuhan 430079, P. R. China

Search for more papers by this author

Tiantian Chen

School of Educational Information Technology, Central China Normal University, Wuhan 430079, P. R. China

Search for more papers by this author

Gang Zhao

School of Educational Information Technology, Central China Normal University, Wuhan 430079, P. R. China

Search for more papers by this author

Mingwen Tong

School of Educational Information Technology, Central China Normal University, Wuhan 430079, P. R. China

Search for more papers by this author

, and

Qingtang Liu

School of Educational Information Technology, Central China Normal University, Wuhan 430079, P. R. China

Search for more papers by this author

https://doi.org/10.1142/S0219691319500012Cited by:8 (Source: Crossref)

Abstract

Observing student body gesture has been widely used to assess teaching effectiveness over the past few decades. However, manual observation is not suitable for the automatic data analysis in the field of learning analytics. Consequently, a student body gesture recognition method based on Fisher Broad Learning System (FBLS) and Local Log-Euclidean Multivariate Gaussian (L²EMG) is proposed in this paper. FBLS is designed by introducing the discriminative information into the hidden layer of Broad Learning System (BLS) and reducing the dimensionality of hidden-layer representations. FBLS has superiorities in accuracy and speed. In addition, L²EMG, which is a highly distinctive descriptor, characterizes the local image with a multivariate Gaussian distribution. So L²EMG features are fed into the FBLS for recognition in this paper. Extensive experimental results on self-built dataset show that the proposed student body gesture recognition method obtains better results than other benchmarking methods.

Keywords:

AMSC: 22E46, 53C35, 57S20

References

1. S. Abidi, M. Piccardi and M.-A. Williams, Static action recognition by efficient greedy inference, in Proc. IEEE Winter Conf. Applications of Computer Vision, Lake Placid, NY, USA, March 7–10 (IEEE, 2016), pp. 1–8. Google Scholar
2. P. Blikstein and M. Worsley, Multimodal learning analytics and education data mining: Using computational technologies to measure complex learning tasks, J. Learn. Anal. 3(2) (2016) 220–238. Crossref, Google Scholar
3. C. Chen, W. Li, H. Su and K. Liu, Spectral-spatial classification of hyperspectral image based on kernel extreme learning machine, Remote Sens. 6(6) (2014) 5795–5814. Crossref, Web of Science, Google Scholar
4. C. P. Chen and Z. Liu, Broad learning system: An effective and efficient incremental learning system without the need for deep architecture, IEEE Trans. Neural Netw. Learn. Syst. 29(1) (2018) 10–24. Crossref, Web of Science, Google Scholar
5. V. Delaitre, I. Laptev and J. Sivic, Recognizing human actions in still images: A study of bag-of-features and part-based representations, in Proc. British Machine Vision Conf., Aberystwyth, Wales, UK, August 30–September 2 (2010), pp. 1–11. Google Scholar
6. P. F. Felzenszwalb, R. B. Girshick, D. McAllester and D. Ramanan, Object detection with discriminatively trained part-based models, IEEE Trans. Pattern Anal. Mach. Intell. 32(9) (2010) 1627–1645. Crossref, Web of Science, Google Scholar
7. R. Ferguson, Learning analytics: Drivers, developments and challenges, Int. J. Technol. Enhanced Learn. 4(5–6) (2012) 304–317. Crossref, Google Scholar
8. S. Herath, M. Harandi and F. Porikli, Going deeper into action recognition: A survey, Image Vis. Comput. 60 (2017) 4–21. Crossref, Web of Science, Google Scholar
9. F. S. Khan, R. M. Anwer, J. Van De Weijer, A. D. Bagdanov, A. M. Lopez and M. Felsberg, Coloring action recognition in still images, Int. J. Comput. Vis. 105(3) (2013) 205–221. Crossref, Web of Science, Google Scholar
10. Y. Kim, T. Soyata and R. F. Behnagh, Towards emotionally-aware ai smart classroom: Current issues and directions for engineering and education, IEEE Access 6 (2018) 5308–5331. Crossref, Web of Science, Google Scholar
11. E. K. Kumar, P. Kishore, A. Sastry, M. T. K. Kumar and D. A. Kumar, Training cnns for 3-d sign language recognition with color texture coded joint angular displacement maps, IEEE Signal Process. Lett. 25(5) (2018) 645–649. Crossref, Web of Science, Google Scholar
12. C. Lang, G. Siemens, A. Wise and D. Gasevic, Handbook of Learning Analytics (SOLAR Society for Learning Analytics and Research, 2017). Crossref, Google Scholar
13. Y. LeCun, Y. Bengio and G. Hinton, Deep learning, Nature 521(7553) (2015) 436–444. Crossref, Web of Science, Google Scholar
14. P. Li, Q. Wang, H. Zeng and L. Zhang, Local log-Euclidean multivariate gaussian descriptor and its application to image classification, IEEE Trans. Pattern Anal. Mach. Intell. 39(4) (2017) 803–817. Crossref, Web of Science, Google Scholar
15. H. Li, Y. Wei, L. Li and C. P. Chen, Hierarchical feature extraction with local neural response for image recognition, IEEE Trans. Cybern. 43(2) (2013) 412–424. Crossref, Web of Science, Google Scholar
16. H. Liu and L. Wang, Gesture recognition for human-robot collaboration: A review, Int. J. Ind. Ergon. (2017). https://doi.org/10.1016/j.ergon.2017.02.004 Web of Science, Google Scholar
17. Z. Luo, B. Peng, D.-A. Huang, A. Alahi and L. Fei-Fei, Unsupervised learning of long-term motion dynamics for videos, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA, July 21–26 (IEEE, 2017), pp. 7101–7110. Google Scholar
18. H. Luo, Y. Y. Tang, R. P. Biuk-Aghai, X. Yang, L. Yang and Y. Wang, Wavelet-based extended morphological profile and deep autoencoder for hyperspectral image classification, Int. J. Wavelets Multiresolut. Inf. Process. 16(03) (2018) 1850016. Link, Web of Science, Google Scholar
19. S. Mitra and T. Acharya, Gesture recognition: A survey, IEEE Trans. Syst. Man Cybern. C Appl. Rev. 37(3) (2007) 311–324. Crossref, Web of Science, Google Scholar
20. G. Mountrakis, J. Im and C. Ogole, Support vector machines in remote sensing: A review, ISPRS J. Photogramm. Remote Sens. 66(3) (2011) 247–259. Crossref, Web of Science, Google Scholar
21. Y.-H. Pao, G.-H. Park and D. J. Sobajic, Learning and generalization characteristics of the random vector functional-link net, Neurocomputing 6(2) (1994) 163–180. Crossref, Web of Science, Google Scholar
22. R. Poppe, A survey on vision-based human action recognition, Image Vis. Comput. 28(6) (2010) 976–990. Crossref, Web of Science, Google Scholar
23. G. A. Rao, K. Syamala, P. Kishore and A. Sastry, Deep convolutional neural networks for sign language recognition, in Proc. IEEE Conf. Signal Processing and Communication Engineering Systems, Vijayawada, India, January 4–5 (IEEE, 2018), pp. 194–197. Google Scholar
24. M. Riesenhuber and T. Poggio, Hierarchical models of object recognition in cortex, Nat. Neurosci. 2(11) (1999) 1019–1025. Crossref, Web of Science, Google Scholar
25. G. Siemens, Learning analytics: The emergence of a discipline, Am. Behav. Sci. 57(10) (2013) 1380–1400. Crossref, Web of Science, Google Scholar
26. J. A. Suykens and J. Vandewalle, Least squares support vector machine classifiers, Neural Process. Lett. 9(3) (1999) 293–300. Crossref, Web of Science, Google Scholar
27. P. Tsarouchi, A. Athanasatos, S. Makris, X. Chatzigeorgiou and G. Chryssolouris, High level robot programming using body and hand gestures, Procedia CIRP 55 (2016) 1–5. Crossref, Google Scholar
28. J. Wang, J. Yang, K. Yu, F. Lv, T. Huang and Y. Gong, Locality-constrained linear coding for image classification, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, San Francisco, CA, USA, June 13–18 (IEEE, 2010), pp. 3360–3367. Google Scholar
29. Y. Xiao, Z. Zhang, A. Beck, J. Yuan and D. Thalmann, Human–Robot interaction by understanding upper body gestures, Presence (Camb) 23(2) (2014) 133–154. Crossref, Google Scholar
30. W. Yang, Y. Wang and G. Mori, Recognizing human actions from still images with latent poses, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, San Francisco, CA, USA, June 13–18 (IEEE, 2010), pp. 2030–2037. Google Scholar
31. J. Yang, K. Yu, Y. Gong and T. Huang, Linear spatial pyramid matching using sparse coding for image classification, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Miami, Florida, June 20–25 (IEEE, 2009), pp. 1794–1801. Google Scholar
32. B. Yao, Understanding human actions in still images, Ph.D. thesis, Stanford University (2013). Google Scholar
33. B. Yao, X. Jiang, A. Khosla, A. L. Lin, L. Guibas and L. Fei-Fei, Human action recognition by learning bases of action attributes and parts, in Proc. IEEE Int. Conf. Computer Vision, Barcelona, Spain, November 6–13 (IEEE, 2011), pp. 1331–1338. Google Scholar
34. S. Yu, Z. Cao and X. Jiang, Robust linear discriminant analysis with a Laplacian assumption on projection distribution, in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, New Orleans, USA, March 5–9 (IEEE, 2017), pp. 2567–2571. Google Scholar
35. M. Yu, J. Xu, J. Zhong, W. Liu and W. Cheng, Behavior detection and analysis for learning process in classroom environment, in Proc. Frontiers in Education Conference, Indianapolis, Indiana, USA, October 18–21 (IEEE, 2017), pp. 1–4. Google Scholar
36. M. D. Zeiler and R. Fergus, Visualizing and understanding convolutional networks, in Proc. European Conf. Computer Vision, Zurich, Switzerland, September 6–12 (Springer, 2014), pp. 818–833. Google Scholar
37. Y. Zhao, X. You, S. Yu, C. Xu, W. Yuan, X.-Y. Jing, T. Zhang and D. Tao, Multi-view manifold learning with locality alignment, Pattern Recognit. 78 (2018) 154–166. Crossref, Web of Science, Google Scholar
38. Z. Zhu, X. You, C. P. Chen, D. Tao, W. Ou, X. Jiang and J. Zou, An adaptive hybrid pattern for noise-robust texture analysis, Pattern Recognit. 48(8) (2015) 2592–2608. Crossref, Web of Science, Google Scholar
39. G. Zhu, L. Zhang, P. Shen and J. Song, Multimodal gesture recognition using 3-d convolution and convolutional lstm, IEEE Access 5 (2017) 4517–4524. Crossref, Web of Science, Google Scholar