Research ArticleNo Access

A Neural Network for Image Anomaly Detection with Deep Pyramidal Representations and Dynamic Routing

Dipartimento di Scienze Matematiche, Informatiche e Fisiche, Università Degli Studi di Udine, Via Delle Scienze 206, 33100 Udine, Italy

E-mail Address: mishra.pankaj@spes.uniud.it

Search for more papers by this author

Claudio Piciarelli

Dipartimento di Scienze Matematiche, Informatiche e Fisiche, Università Degli Studi di Udine, Via Delle Scienze 206, 33100 Udine, Italy

E-mail Address: claudio.piciarelli@uniud.it

Corresponding author.

Search for more papers by this author

, and

Gian Luca Foresti

Dipartimento di Scienze Matematiche, Informatiche e Fisiche, Università Degli Studi di Udine, Via Delle Scienze 206, 33100 Udine, Italy

E-mail Address: gianluca.foresti@uniud.it

Search for more papers by this author

https://doi.org/10.1142/S0129065720500604Cited by:30 (Source: Crossref)

Abstract

Image anomaly detection is an application-driven problem where the aim is to identify novel samples, which differ significantly from the normal ones. We here propose Pyramidal Image Anomaly DEtector (PIADE), a deep reconstruction-based pyramidal approach, in which image features are extracted at different scale levels to better catch the peculiarities that could help to discriminate between normal and anomalous data. The features are dynamically routed to a reconstruction layer and anomalies can be identified by comparing the input image with its reconstruction. Unlike similar approaches, the comparison is done by using structural similarity and perceptual loss rather than trivial pixel-by-pixel comparison. The proposed method performed at par or better than the state-of-the-art methods when tested on publicly available datasets such as CIFAR10, COIL-100 and MVTec.

Keywords:

References

1. M. L. Antonie, O. R. Zaïane and A. Coman , Application of data mining techniques for medical image classification, in Proc. 2nd Int. Conf. Multimedia Data Mining (San Francisco, CA, USA, 2001), pp. 94–101. Google Scholar
2. M. Ahmed, A. N. Mahmood and J. Hu , A survey of network anomaly detection techniques, J. Network Comput. Appl. 60 (2016) 19–31. Crossref, Web of Science, Google Scholar
3. C. Piciarelli, D. Avola, D. Pannone and G. L. Foresti , A vision-based system for internal pipeline inspection, IEEE Trans. Ind. Inf. 15(6) (2019) 3289–3299. Crossref, Web of Science, Google Scholar
4. P. Chen, S. Yang and J. A. McCann , Distributed real-time anomaly detection in networked industrial sensing systems, IEEE Trans. Ind. Electron. 62(6) (2015) 3832–3842. Crossref, Web of Science, Google Scholar
5. C. Piciarelli, C. Micheloni and G. L. Foresti , Trajectory-based anomalous event detection, IEEE Trans. Circuits Syst. Video Technol. 18(11) (2008) 1544–1554. Crossref, Web of Science, Google Scholar
6. A. M. Soares, B. J. Fernandes and C. J. Bastos-Filho , Structured pyramidal neural networks, Int. J. Neural Syst. 28(5) (2018) 1750021. Link, Web of Science, Google Scholar
7. S. Sabour, N. Frosst and G. E. Hinton , Dynamic routing between capsules, in Advances in Neural Information Processing Systems (Long Beach, CA, USA, 2017), pp. 3856–3866. Google Scholar
8. L. Deecke, R. Vandermeulen, L. Ruff, S. Mandt and M. Kloft , Image anomaly detection with generative adversarial networks, in European Conf. Machine Learning and Knowledge Discovery in Databases (Dublin, Ireland, 2018), pp. 3–17. Google Scholar
9. S. Akcay, A. Atapour–Abarghouei and T. Breckon , Ganomaly: Semi-supervised anomaly detection via adversarial training, in Proc. Asian Conf. Computer Vision (Perth, Australia, 2018), pp. 622–637. Google Scholar
10. C. Zhou and R. C. Paffenroth , Anomaly detection with robust deep autoencoders, in Proc. 23rd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining (New York, NY, USA, 2017), pp. 665–674. Crossref, Google Scholar
11. L. Ruff, R. Vandermeulen, N. Goernitz, L. Deecke, S. A. Siddiqui, A. Binder, E. Müller and M. Kloft, Deep one-class classification, in Proceedings of the 35th Int. Conf. Machine Learning, eds. J. Dy and A. Krause Proceedings of Machine Learning Research 80 (PMLR, Stockholmsmässan, Stockholm Sweden, 2018), pp. 4393–4402. Google Scholar
12. P. Perera, R. Nallapati and B. Xiang , Ocgan: One-class novelty detection using gans with constrained latent representations, in Proc. IEEE Conf. Computer Vision and Pattern Recognition (Long Beach, CA, USA, 2019), pp. 2898–2906. Crossref, Google Scholar
13. Y. LeCun, L. Bottou, Y. Bengio, P. Haffner et al., Gradient-based learning applied to document recognition, in Proc. IEEE 86(11) (1998) 2278–2324. Crossref, Web of Science, Google Scholar
14. P. Bergmann, M. Fauser, D. Sattlegger and C. Steger , MVTec AD–a comprehensive real-world dataset for unsupervised anomaly detection, in Proc. IEEE Conf. Computer Vision and Pattern Recognition (Long Beach, CA, USA, 2019), pp. 9592–9600. Crossref, Google Scholar
15. D. Carrera, F. Manganini, G. Boracchi and E. Lanzarone , Defect detection in sem images of nanofibrous materials, IEEE Trans. Indust. Inf. 13(2) (2017) 551–561. Crossref, Web of Science, Google Scholar
16. V. Chandola, A. Banerjee and V. Kumar , Anomaly detection: A survey, ACM Comput. Surveys 41(3) (2009) 15:1–15:58. Crossref, Web of Science, Google Scholar
17. M. A. Pimentel, D. A. Clifton, L. Clifton and L. Tarassenko , A review of novelty detection, Signal Process. 99 (2014) 215–249. Crossref, Web of Science, Google Scholar
18. I. Goodfellow, Y. Bengio and A. Courville , Deep Learning (MIT Press, 2016), http://www.deeplearningbook.org. Google Scholar
19. A. Krizhevsky, I. Sutskever and G. E. Hinton , Imagenet classification with deep convolutional neural networks, Advances in Neural Information Processing Systems (Lake Tahoe, NV, USA, 2012), pp. 1097–1105. Google Scholar
20. O. M. Manzanera, S. K. Meles, K. L. Leenders, R. J. Renken, M. Pagani, D. Arnaldi, F. Nobili, J. Obeso, M. R. Oroz, S. Morbelli et al., Scaled subprofile modeling and convolutional neural networks for the identification of parkinson’s disease in 3d nuclear imaging data, Int. J. Neural Systems 29(9) (2019) 1950010. Link, Web of Science, Google Scholar
21. A. H. Ansari, P. J. Cherian, A. Caicedo, G. Naulaers, M. De Vos and S. Van Huffel , Neonatal seizure detection using deep convolutional neural networks, Int. J. Neural Systems 29(4) (2019) 1850011. Link, Web of Science, Google Scholar
22. C. Hua, H. Wang, H. Wang, S. Lu, C. Liu and S. M. Khalid , A novel method of building functional brain network using deep learning algorithm with application in proficiency detection, Int. J. Neural Syst. 29(1) (2019) 1850015. Link, Web of Science, Google Scholar
23. A. Antoniades, L. Spyrou, D. Martin-Lopez, A. Valentin, G. Alarcon, S. Sanei and C. C. Took , Deep neural architectures for mapping scalp to intracranial eeg, Int. J. Neural Syst. 28(8) (2018) 1850009. Link, Web of Science, Google Scholar
24. U. R. Acharya, S. L. Oh, Y. Hagiwara, J. H. Tan and H. Adeli , Deep convolutional neural network for the automated detection and diagnosis of seizure using eeg signals, Comput. Biol. Med. 100 (2018) 270–278. Crossref, Medline, Web of Science, Google Scholar
25. U. R. Acharya, S. L. Oh, Y. Hagiwara, J. H. Tan, H. Adeli and D. P. Subha , Automated eeg-based screening of depression using deep convolutional neural network, Comput. Methods Programs Biomed. 161 (2018) 103–113. Crossref, Medline, Web of Science, Google Scholar
26. M. H. Rafiei and H. Adeli , A new neural dynamic classification algorithm, IEEE Trans. Neural Networks Learn. Syst. 28(12) (2017) 3074–3083. Crossref, Medline, Web of Science, Google Scholar
27. B. R. Kiran, D. M. Thomas and R. Parakkal , An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos, J. Imaging 4(2) (2018) 1–25. Crossref, Web of Science, Google Scholar
28. A. Géron , Hands-on Machine Learning with Scikit–Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems (O’Reilly Media, 2017). Google Scholar
29. C. Ieracitano, F. Pantó, N. Mammone, A. Paviglianiti, P. Frontera and F. C. Morabito, Toward an Automatic Classification of SEM Images of Nanomaterials via a Deep Learning Approach, Neural Approaches to Dynamics of Signal Exchanges eds. A. Esposito, M. Faundez-Zanuy, F. C. Morabito and E. Pasero (Springer, Singapore, 2020), pp. 61–72. Google Scholar
30. M. Buda, A. Maki and M. A. Mazurowski , A systematic study of the class imbalance problem in convolutional neural networks, Neural Networks 106 (2018) 249–259. Crossref, Medline, Web of Science, Google Scholar
31. S. K. Lim, Y. Loo, N.-T. Tran, N.-M. Cheung, G. Roig and Y. Elovici , Doping: Generative data augmentation for unsupervised anomaly detection with GAN, 2018 IEEE Int. Conf. Data Mining (ICDM) (Singapore, 2018), pp. 1122–1127. Crossref, Google Scholar
32. C. Piciarelli, P. Mishra and G. L. Foresti , Image anomaly detection with capsule networks and imbalanced datasets, Int. Conf. Image Analysis and Processing (Trento, Italy, 2019), pp. 257–267. Crossref, Google Scholar
33. P. Perera and V. M. Patel , Learning deep features for one-class classification, IEEE Trans. Image Process. 28 (2019) 5450–5463. Crossref, Medline, Web of Science, Google Scholar
34. S. M. Erfani, S. Rajasegarar, S. Karunasekera and C. Leckie , High-dimensional and large-scale anomaly detection using a linear one-class svm with deep learning, Pattern Recognit. 58 (2016) 121–134. Crossref, Web of Science, Google Scholar
35. M. Gutoski, N. M. R. Aquino, M. Ribeiro, E. Lazzaretti and H. S. Lopes , Detection of video anomalies using convolutional autoencoders and one-class support vector machines, in XIII Brazilian Congress on Computational Intelligence (Curitiba, Brazil, 2017), pp. 1–12. Google Scholar
36. C. Aytekin, X. Ni, F. Cricri and E. Aksu , Clustering and unsupervised anomaly detection with L2 normalized deep auto-encoder representations, 2018 Int. Joint Conf. Neural Networks (IJCNN) (Rio, Brazil, 2018), pp. 1–6. Crossref, Google Scholar
37. Z. Ghafoori and C. Leckie , Deep multi-sphere support vector data description, Proc. 2020 SIAM Int. Conf. Data Mining (SIAM, Cincinnati, OH, USA, 2020), pp. 109–117. Crossref, Google Scholar
38. R. Chalapathy, A. K. Menon and S. Chawla, Anomaly detection using one-class neural networks, arXiv:1802.06360. Google Scholar
39. M. Nadeem, O. Marshall, S. Singh, X. Fang and X. Yuan , Semi-supervised deep neural network for network intrusion detection, 2016 KSU Conf. on Cybersecurity Education, Research and Practice (Kennesaw, GA, USA, 2016), pp. 1–11. Google Scholar
40. D. Abati, A. Porrello, S. Calderara and R. Cucchiara , Latent space autoregression for novelty detection, in Proc. IEEE Conf. Comp. Vision and Pattern Recognition (Long Beach, CA, USA, 2019), pp. 481–490. Crossref, Google Scholar
41. T. Schlegl, P. Seeböck, S. M. Waldstein, U. Schmidt-Erfurth and G. Langs , Unsupervised anomaly detection with generative adversarial networks to guide marker discovery, Int. Conf. Information Processing in Medical Imaging (Boone, NC, USA, 2017), pp. 146–157. Crossref, Google Scholar
42. P. Oza and V. M. Patel , One-class convolutional neural network, IEEE Signal Process. Lett. 26(2) (2018) 277–281. Crossref, Web of Science, Google Scholar
43. P. Bergmann, S. Löwe, M. Fauser, D. Sattlegger and C. Steger, Improving unsupervised defect segmentation by applying structural similarity to autoencoders, arXiv:1807.02011. Google Scholar
44. M. Sabokrou, M. Khalooei, M. Fathy and E. Adeli , Adversarially learned one-class classifier for novelty detection, in Proc. IEEE Conf. Computer Vision and Pattern Recognition (Salt Lake City, UT, USA, 2018), pp. 3379–3388. Crossref, Google Scholar
45. J. Chen, S. Sathe, C. Aggarwal and D. Turaga , Outlier detection with autoencoder ensembles, in Proc. 2017 SIAM Int. Conference on Data Mining, SIAM, 2017, pp. 90–98. Crossref, Google Scholar
46. Y. Xia, X. Cao, F. Wen, G. Hua and J. Sun , Learning discriminative reconstructions for unsupervised outlier removal, in Proc. IEEE Int. Conf. Computer Vision, 2015, pp. 1511–1519. Crossref, Google Scholar
47. J. Sun, X. Wang, N. Xiong and J. Shao , Learning sparse representation with variational auto-encoder for anomaly detection, IEEE Access 6 (2018) 33353–33361. Crossref, Web of Science, Google Scholar
48. J. Hu, L. Shen and G. Sun , Squeeze-and-excitation networks, in 2018 IEEE/CVF Conf. Computer Vision and Pattern Recognition (Salt Lake City, UT, USA, 2018), pp. 7132–7141. Crossref, Google Scholar
49. J. Johnson, A. Alahi and L. Fei-Fei , Perceptual losses for real-time style transfer and super-resolution, European Conf. Computer Vision (Amsterdam, The Netherlands, 2016), pp. 694–711. Crossref, Google Scholar
50. P. Bergmann, S. Löwe, M. Fauser, D. Sattlegger and C. Steger , Improving unsupervised defect segmentation by applying structural similarity to autoencoders, in Int. Joint Conf. Computer Vision, Imaging and Computer Graphics Theory and Applications (Prague, Czech Republic, 2019), pp. 372–380. Crossref, Google Scholar
51. A. Krizhevsky, Learning multiple layers of features from tiny images, Master’s thesis, Department of Computer Science, University of Toronto (2009). Google Scholar
52. S. A. Nene, S. K. Nayar and H. Murase, Columbia object image library (COIL-100), Tech. Report Technical Report CUCS-006-96, Columbia University (1996). Google Scholar
53. S. Pidhorskyi, R. Almohsen and G. Doretto , Generative probabilistic novelty detection with adversarial autoencoders, Advances in Neural Information Processing Systems (Montréal, Canada, 2018), pp. 6822–6833. Google Scholar
54. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai and S. Chintala , Pytorch: An imperative style, high-performance deep learning library, Advances in Neural Information Processing Systems 32, eds. H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E. Fox and R. Garnett (Curran Associates, Inc., 2019), pp. 8024–8035. Google Scholar
55. D. P. Kingma and M. Welling , Auto-encoding variational bayes, Int. Conf. Learning Representations (Banff, Canada, 2014). Google Scholar
56. A. Van den Oord, N. Kalchbrenner, L. Espeholt, O. Vinyals, A. Graves et al., Conditional image generation with pixelcnn decoders, Advances in Neural Information Processing Systems (Barcelona, Spain, 2016), pp. 4790–4798. Google Scholar
57. M. Sakurada and T. Yairi , Anomaly detection using autoencoders with nonlinear dimensionality reduction, in Proc. MLSDA 2014 2nd Workshop on Machine Learning for Sensory Data Analysis, MLSDA’14, (Association for Computing Machinery, New York, NY, USA, 2014), pp. 4–11. Crossref, Google Scholar
58. P. Napoletano, F. Piccoli and R. Schettini , Anomaly detection in nanofibrous materials by CNN-based self-similarity, Sensors 18(1) (2018) p. 209. Crossref, Web of Science, Google Scholar