No Access

A Method of Environmental Sound Classification Based on Residual Networks and Data Augmentation

Jinfang Zeng

School of Physics and Optoelectronics, Xiang Tan University, Xiangtan, Hunan Province 411105, P. R. China

E-mail Address: jinfangzeng123@gmail.com

Corresponding author.

Search for more papers by this author

Youming Li

School of Physics and Optoelectronics, Xiang Tan University, Xiangtan, Hunan Province 411105, P. R. China

Search for more papers by this author

Yu Zhang

School of Physics and Optoelectronics, Xiang Tan University, Xiangtan, Hunan Province 411105, P. R. China

Search for more papers by this author

, and

Da Chen

School of Physics and Optoelectronics, Xiang Tan University, Xiangtan, Hunan Province 411105, P. R. China

Search for more papers by this author

https://doi.org/10.1142/S1469026821500188Cited by:0 (Source: Crossref)

Abstract

Environmental sound classification (ESC) is a challenging problem due to the complexity of sounds. To date, a variety of signal processing and machine learning techniques have been applied to ESC task, including matrix factorization, dictionary learning, wavelet filterbanks and deep neural networks. It is observed that features extracted from deeper networks tend to achieve higher performance than those extracted from shallow networks. However, in ESC task, only the deep convolutional neural networks (CNNs) which contain several layers are used and the residual networks are ignored, which lead to degradation in the performance. Meanwhile, a possible explanation for the limited exploration of CNNs and the difficulty to improve on simpler models is the relative scarcity of labeled data for ESC. In this paper, a residual network called EnvResNet for the ESC task is proposed. In addition, we propose to use audio data augmentation to overcome the problem of data scarcity. The experiments will be performed on the ESC-50 database. Combined with data augmentation, the proposed model outperforms baseline implementations relying on mel-frequency cepstral coefficients and achieves results comparable to other state-of-the-art approaches in terms of classification accuracy.

Keywords:

Remember to check out the Most Cited Articles!
Check out these titles in artificial intelligence!

References

1. V. Bisot, R. Serizel, S. Essid and G. Richard, Feature learning with matrix factorization applied to acoustic scene classification, IEEE/ACM Trans. Audio, Speech, Language Process. 25(6) (2017) 1216–1229. Crossref, Google Scholar
2. B. Elizalde, H. Lei, G. Friedland and N. Peters, An i-vector based approach for audio scene detection, IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events (2013), pp. 114–117. Google Scholar
3. J. Salamon and J. P. Bello, Deep convolutional neural networks and data augmentation for environmental sound classification, IEEE Signal Process. Lett. 24(3) (2017) 279–283. Crossref, Google Scholar
4. Z. Zhang et al., Attention based convolutional recurrent neural network for environmental sound classification. ArXiv abs/1907.02230 (2019): n. page. Google Scholar
5. K. J. Piczak, Environmental sound classification with convolutional neural networks, in 25th Int. Workshop on Machine Learning for Signal Processing (MLSP), Sep. 2015, Boston, MA, USA, pp. 1–6. Google Scholar
6. C.-Y. Lee, S. Xie, P. Gallagher, Z. Zhang and Z. Tu, Deeply-supervised nets, in Proc. AISTATS, 2015, pp. 562–570. Google Scholar
7. J. T. Springenberg, A. Dosovitskiy, T. Brox and M. Riedmiller, Striving for simplicity: The all convolutional net, preprint (2014) arXiv:1412.6806. Google Scholar
8. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke and A. Rabinovich, Going deeper with convolutions, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 1–9. Crossref, Google Scholar
9. K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, preprint (2015) arXiv:1512.03385. Google Scholar
10. K. Zhang et al., Residual networks of residual networks: multilevel residual networks, IEEE Trans. Circuits Syst. Video Technol. 28(6) (2018) 1303–1314. Crossref, Google Scholar
11. J. Salamon, C. Jacoby and J. P. Bello, A dataset and taxonomy for urban sound research, in 22nd ACM Int. Conf. Multimedia (ACM-MM’14), Orlando, FL, USA, November 2014, pp. 1041–1044. Google Scholar
12. K. J. Piczak, ESC: Dataset for environmental sound classification, in 23rd ACM Int. Conf. on Multimedia, Brisbane, Australia, Oct. 2015, pp. 1015–1018. Google Scholar
13. A. Mesaros, E. Fagerlund, A. Hiltunen, T. Heittola and T. Virtanen, TUT sound events 2016, development dataset (2016), http://dx.doi.org/10.5281/zenodo.45759. Google Scholar
14. A. Krizhevsky, I. Sutskever and G. Hinton, ImageNet classification with deep convolutional neural networks, in Advances in Neural Information Processing Systems (NIPS), 2012, pp. 1097–1105. Google Scholar
15. P. Y. Simard, D. Steinkraus and J. C. Platt, Best practices for convolutional neural networks applied to visual document analysis, in Int. Conf. Document Analysis and Recognition, Vol. 3, Edinburgh, Scottland, UK, Aug. 2003, pp. 958–962. Google Scholar
16. B. McFee, E. Humphrey and J. Bello, A software framework for musical data augmentation, in 16th Int. Society for Music Information Retrieval Conf., Malaga, Spain, Oct. 2015, pp. 248–254. Google Scholar
17. A. Mesaros, T. Heittola, O. Dikmen and T. Virtanen, Sound event Detection in real life recordings using coupled matrix factorization of Spectral representations and class activity annotations, in Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), Brisbane, Australia, Apr. 2015, pp. 151–155. Google Scholar
18. E. Benetos, G. Lafay, M. Lagrange and M. D. Plumbley, Detection of Overlapping acoustic events using a temporally-constrained probabilistic Model, in IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China, Mar. 2016, pp. 6450–6454. Google Scholar
19. V. Bisot, R. Serizel, S. Essid and G. Richard, Acoustic scene classification with matrix factorization for unsupervised feature learning, in IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China, Mar. 2016, pp. 6445–6449. Google Scholar
20. J. Salamon and J. P. Bello, Unsupervised feature learning for urban Sound classification, in IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), Brisbane, Australia, Apr. 2015, pp. 171–175. Google Scholar
21. J. Salamon and J. P. Bello, Feature learning with deep scattering for urban sound analysis, 2015 23rd European Signal Processing Conference (EUSIPCO), Nice, 2015, pp. 724–728. Crossref, Google Scholar
22. J. T. Geiger and K. Helwani, Improving event detection for audio Surveillance using gabor filterbank features, in 23rd European Signal Processing Conference (EUSIPCO), Nice, France, Aug. 2015, pp. 714–718. Google Scholar
23. E. Cakir, T. Heittola, H. Huttunen and T. Virtanen, Polyphonic sound Event detection using multi label deep neural networks, in 2015 Int. Joint Conf. Neural Networks (IJCNN), July 2015, pp. 1–7. Google Scholar
24. D. Giannoulis, E. Benetos, D. Stowell, M. Rossignol, M. Lagrange and M. D. Plumbley, Detection and classification of acoustic scenes and events: An IEEE AASP challenge, in IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA, Oct. 2013, pp. 1–4. Google Scholar
25. D. Stowell, D. Giannoulis, E. Benetos, M. Lagrange and M. D. Plumbley, Detection and classification of acoustic scenes and events, IEEE Trans. Multi. 17(10) (2015) 1733–1746. Crossref, Google Scholar
26. S. Sigtia, A. Stark, S. Krstulovic and M. Plumbley, Automatic environmental sound recognition: Performance versus computational cost, IEEE/ACM Trans. Audio SpeechLang. Process. PP(99) (2016) 1. Google Scholar
27. Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE 86(11) (1998) 2278–2324. Crossref, Google Scholar
28. F. Chollet, Keras. https://github.com/fchollet/keras ${$ ${$ Last accessed on 26th February, 2017 $}$ $}$ . Google Scholar
29. R. N. Tak, D. M. Agrawal and H. A. Patil, Novel phase encoded mel filterbank energies for environmental sound classification, in Int. Conf. Pattern Recognition and Machine Intelligence (Springer, Cham, 2017), pp. 317–325. Crossref, Google Scholar
30. B. Zhu, C. Wang, F. Liu, J. Lei, Z. Lu and Y. Peng, Learning environmental sounds with multi-scale convolutional neural network, preprint (2018), arXiv:1803.10219. Google Scholar
31. Y. Tokozume and T. Harada, Learning environmental sound with end-to-end convolutional neural network, in IEEE Int. Conf. Acoustics, Speech and Signal Process (ICASSP), New Orleans, USA, 2017, pp. 2721–2725. Crossref, Google Scholar