Research PaperNo Access

A Sentiment Analysis Anomaly Detection System for Cyber Intelligence

Computer Science Centre, University of Messina, Piazza Antonello, 2, 98122 Messina, Italy

E-mail Address: roberta.maisano@unime.it

Corresponding author.

and

Department of Mathematics, Computer Science and Physics, University of Udine, Viale delle Scienze, 206, 33100 Udine, Italy

Search for more papers by this author

https://doi.org/10.1142/S012906572350003XCited by:5 (Source: Crossref)

Abstract

Considering the 2030 United Nations intent of world connection, Cyber Intelligence becomes the main area of the human dimension able of inflicting changes in geopolitical dynamics. In cyberspace, the new battlefield is the mind of people including new weapons like abuse of social media with information manipulation, deception by activists and misinformation. In this paper, a Sentiment Analysis system with Anomaly Detection (SAAD) capability is proposed. The system, scalable and modular, uses an OSINT-Deep Learning approach to investigate on social media sentiment in order to predict suspicious anomaly trend in Twitter posts. Anomaly detection is investigated with a new semi-supervised process that is able to detect potentially dangerous situations in critical areas. The main contributions of the paper are the system suitability for working in different areas and domains, the anomaly detection procedure in sentiment context and a time-dependent confusion matrix to address model evaluation with unbalanced dataset. Real experiments and tests were performed on Sahel Region. The detected anomalies in negative sentiment have been checked by experts of Sahel area, proving true links between the models results and real situations observable from the tweets.

Keywords:

References

1. W. Guo, K. Gleditsch and A. Wilson, Retool AI to forecast and limit wars, Nature 562 (2018) 331–333. Crossref, Medline, Web of Science, Google Scholar
2. H. S. Nogay and H. Adeli, Detection of epileptic seizure using pre-trained deep convolutional neural network and transfer learning, Eur. Neurol. 83(6) (2020) 602–614. Crossref, Medline, Web of Science, Google Scholar
3. U. R. Acharya, S. Vidya, H. Adeli, S. Jayashree, J. E. W. Koh and A. Adeli, Computer aided diagnosis of depression using EEG signals, Eur. Neurol. 73 (2015) 329–336. Crossref, Medline, Web of Science, Google Scholar
4. G. B. Martins, J. P. Papa and H. Adeli, Deep learning techniques for recommender systems based on collaborative filtering, Expert Syst. 37(6) (2020) e12647. Crossref, Web of Science, Google Scholar
5. G. L. Foresti, From pioneering artificial neural networks to deep learning and beyond, Int. J. Neural Syst. 31(5) (2021) 2103004. Link, Web of Science, Google Scholar
6. A. Pak and P. Paroubek, Twitter as a corpus for sentiment analysis and opinion mining, in Proc. 7th Int. Conf. Language Resources and Evaluation (LREC 10), 2010, Valletta, Malta, pp. 1320–1326. Google Scholar
7. A. Go, R. Bhayani and L. Huang, Twitter sentiment classification using distant supervision, CS224N Project Report, Stanford, Vol. 1 (2009), p. 12. Google Scholar
8. J. Wehrmann, W. Becker, H. E. Cagnini and R. C. Barros, A character-based convolutional neural network for language-agnostic Twitter sentiment analysis, in Proc. 2017 Int. Joint Conf. Neural Networks (IJCNN), 2017, Anchorage, AK, USA, pp. 2384–2391. Crossref, Google Scholar
9. H. S. Nogay and H. Adeli, Machine learning (ML) for the diagnosis of autism spectrum disorder (ASD) using brain imaging, Rev. Neurosci. 31 (2020) 825–841, https://doi.org/10.1515/revneuro-2020-0043. Crossref, Web of Science, Google Scholar
10. D. Stojanovski, G. Strezoski, G. Madjarov and I. Dimitrovski, Twitter sentiment analysis using deep convolutional neural network, 10th International Conference in Hybrid Artificial Intelligent Systems, HAIS, Bilbao, Spain (2015), LNCS (LNAI), vol. 9121, pp. 726–737. Springer, Cham., doi:10.1007/978-3-319-19644-260. Google Scholar
11. P. M. Sosa, Twitter sentiment analysis using combined LSTM-CNN models (2017) Eprint Arxiv, pp. 1–9. Google Scholar
12. N. Chen and P. Wang, Advanced combined LSTM-CNN model for twitter sentiment analysis, in 5th IEEE Int. Conf. Cloud Computing and Intelligence Systems (CCIS) (Nanjing, Jiangsu, China, 2018), pp. 684–687, https://doi.org/10.1109/CCIS.2018.8691381. Crossref, Google Scholar
13. S. Minaee, E. Azimi and A. Abdolrashidi, Deep-sentiment: Sentiment analysis using ensemble of CNN and Bi-LSTM models, arXiv:abs/1904.04206. Google Scholar
14. S. C. Nistor, M. Moca, D. Moldovan, D. B. Oprean and R. L. Nistor, Building a twitter sentiment analysis system with recurrent neural networks, Sensors 21(7) (2021) 2266, https://doi.org/10.3390/s21072266. Crossref, Web of Science, Google Scholar
15. A. S. M. Alharbi and E. de Doncker, Twitter sentiment analysis with a deep neural network: An enhanced approach using user behavioral information, Cogn. Syst. Res. 54 (2019) 50–61. Crossref, Web of Science, Google Scholar
16. M. Pota, M. Ventura, R. Catelli and M. Esposito, An effective BERT-based pipeline for twitter sentiment analysis: A case study in Italian, Sensors 21(1) (2021) 133, https://doi.org/10.3390/s21010133. Crossref, Web of Science, Google Scholar
17. K. Sarkar, Sentiment polarity detection in Bengali tweets using deep convolutional neural networks, J. Intell. Syst. 28(3) (2019) 377–386, https://doi.org/10.1515/jisys-2017-0418. Web of Science, Google Scholar
18. B. Alharbi, H. Alamro, M. Alshehri, Z. Khayyat, M. Kalkatawi, I. I. Jaber and X. Zhang, ASAD: A twitter-based benchmark arabic sentiment analysis dataset (2020), arxiv.2011.00578. Google Scholar
19. S. Liao, J. Wang, R. Yu, K. Sato and Z. Cheng, CNN for situations understanding based on sentiment analysis of Twitter data, Procedia Comput. Sci. 111 (2017) 376–381, https://doi.org/10.1016/j.procs.2017.06.037. Crossref, Google Scholar
20. P. Barnaghi, P. Ghaffari and J. G. Breslin, Opinion mining and sentiment polarity on twitter and correlation between events and sentiment, in 2016 IEEE Second Int. Conf. Big Data Computing Service and Applications (BigDataService) (Oxford, United Kingdom, 2016), pp. 52–57, https://doi.org/10.1109/BigDataService.2016.36. Crossref, Google Scholar
21. M. Vernier, M. Farinosi and G. L. Foresti, A smart visual information tool for situational awareness, in Proc. 11th Joint Conf. Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP), Vol. 3: VISAPP, 2016, pp. 236–245, https://doi.org/10.5220/0005680402360245. Crossref, Google Scholar
22. M. Vernier, M. Farinosi and G. L. Foresti, Twitter data mining for situational awareness, in Advanced Methodologies and Technologies in Network Architecture, Mobile Computing, and Data Analytics, ed. D. B. A. Mehdi Khosrow-Pour (IGI Global, 2019), pp. 684–695, https://doi.org/10.4018/978-1-5225-7598-6.ch050. Crossref, Google Scholar
23. A. Hernndez, V. Sanchez, G. Sanchez, H. Perez-Meana, J. Olivares Mercado, K. Toscano, M. Nakano and V. Martinez, Security attack prediction based on user sentiment analysis of Twitter data, in 2016 IEEE Int. Conf. Industrial Technology (ICIT) (2016), pp. 610–617, https://doi.org/10.1109/ICIT.2016.7474819. Crossref, Google Scholar
24. G. Alexandridis, J. Aliprantis, K. Michalakis, K. Korovesis, P. Tsantilas and G. Caridakis, A knowledge-based deep learning architecture for aspect-based sentiment analysis, Int. J. Neural Syst. 31(10) (2021) 2150046. Link, Web of Science, Google Scholar
25. P. Hajek, A. Barushka and M. Munk, Neural networks with emotion associations, topic modeling and supervised term weighting for sentiment analysis, Int. J. Neural Syst. 31(10) (2021) 2150013. Link, Web of Science, Google Scholar
26. P. Mishra, C. Piciarelli and G. L. Foresti, A neural network for image anomaly detection with deep pyramidal representations and dynamic routing, Int. J. Neural Syst. 30(10) (2020) 2050060. Link, Web of Science, Google Scholar
27. Z. Wang, V. Joo, C. Tong, X. Xin and H. C. Chin, Anomaly detection through enhanced sentiment analysis on social media data, in 2014 IEEE 6th Int. Conf. Cloud Computing Technology and Science (Singapore, 2014), pp. 917–922, https://doi.org/10.1109/CloudCom.2014.69. Crossref, Google Scholar
28. S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Comput. 9(8) (1997) 1735–1780, https://doi.org/10.1162/neco.1997.9.8.1735. Crossref, Medline, Web of Science, Google Scholar
29. B. Bhavitha, A. P. Rodrigues and N. N. Chiplunkar, Comparative study of machine learning techniques in sentimental analysis, in Proc. 2017 Int. Conf. Inventive Communication and Computational Technologies (ICICCT) (Coimbatore, India, 2017), pp. 216–221. Crossref, Google Scholar
30. M. Ester, H. P. Kriegel, J. Sander and X. Xu, A density-based algorithm for discovering clusters in large spatial databases with noise, in Proc. Second Int. Conf. Knowledge Discovery and Data Mining (KDD’96) (AAAI Press, 1996), pp. 226–231. Google Scholar
31. T. Mikolov, K. Chen, G. Corrado and J. Dean, Efficient estimation of word representations in vector space, in Proc. Workshop at Int. Conf. Learning Representations (Scottsdale, AZ, USA, 2013), DBLP:journals/corr/abs-1301-3781 Google Scholar
32. F. Poldi and C. Zacharias (2019) TWINT Project Open-source Twitter Intelligence [Source code]. https://github.com/twintproject. Google Scholar
33. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard and L. D. Jackel, Backpropagation applied to handwritten zip code recognition, Neural Comput. 1(4) (1989) 541–551. Crossref, Web of Science, Google Scholar
34. A. Graves, A. R. Mohamed and G. Hinton, Speech recognition with deep recurrent neural networks, in 2013 IEEE Int. Conf. Acoustics, Speech and Signal Processing (2013), pp. 6645–6649. Crossref, Google Scholar
35. A. Gulli and S. Pal (2017). Deep learning with Keras. Packt Publishing Ltd. Google Scholar
36. A. Hassan and A. Mahmood, Deep learning approach for sentiment analysis of short texts, in Proc. Third Int. Conf. Control, Automation and Robotics (ICCAR), 24–26 April 2017, Nagoya, Japan, pp. 705–710. Crossref, Google Scholar
37. R. Rehurek and P. Sojka, Software Framework for Topic Modelling with Large Corpora, in Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. Valletta, Malta: ELRA, May (2010), pp. 45–50. Google Scholar
38. J. P. Rousseeuw, Silhouettes: A graphical aid to the interpretation and validation of cluster analysis, J. Comput. Appl. Math. 20 (1987) 53–65, https://doi.org/10.1016/0377-0427(87)90125-7. Crossref, Web of Science, Google Scholar
39. C.-Y. Liou, W.-C. Cheng, J.-W. Liou and D.-R. Liou, Autoencoder for words, Neurocomputing 139 (2014) 84–96. Crossref, Web of Science, Google Scholar
40. G. Cicceri, R. Maisano, N. Morey and S. Distefano, A machine learning approach for anomaly detection in environmental IoT-driven wastewater purification systems, World Academy of Science, Engineering and Technology, Open Science Index 171, Int. J. Environ. Ecological Eng. 15(3) (2021) 123–130. Google Scholar
41. P. Branco, L. Torgo and R. Ribeiro, A survey of predictive modeling on imbalanced domains, ACM Comput. Surv. 49(2) (2017) 31, https://doi.org/10.1145/2907070. Crossref, Web of Science, Google Scholar
42. X. Zhou and A. Del Valle, Range based confusion matrix for imbalanced time series classification, in 6th Conf. Data Science and Machine Learning Applications (CDMA) (Riyadh, Saudi Arabia, 2020), pp. 1–6, https://doi.org/10.1109/CDMA47397.2020.00006. Crossref, Google Scholar
43. H. Saif, M. Fernandez and H. Alani, Evaluation datasets for twitter sentiment analysis. A survey and a new dataset, the STS-Gold, in 1st International Workshop on Emotion and Sentiment in Social and Expressive Media: Approaches and Perspectives from AI (ESSEM 2013), 3 December, Turin, Italy (2013), pp. 9–21. Google Scholar
44. S. Loria, textblob Documentation. Release 0.15, 2. (2018). Google Scholar
45. F. Abid, M. Alam, M. Yasir and C. Li, Sentiment analysis through recurrent variants latterly on convolutional neural network of Twitter, Future Gener. Comput. Syst. 95 (2019) 292–308, https://doi.org/10.1016/j.future.2018.12.018. Crossref, Web of Science, Google Scholar
46. C. N. Dang, M. N. Moreno-Garcia and F. De la Prieta, Hybrid deep learning models for sentiment analysis, Hindawi Complexity 2021 (2021) 16, https://doi.org/10.1155/2021/9986920. Web of Science, Google Scholar
47. K. Han, W. Chien, C. Chiu and Y. Cheng, Application of support vector machine (SVM) in the sentiment analysis of twitter dataset, Appl. Sci. 10(3) (2020) 1125, https://doi.org/10.3390/app10031125. Crossref, Google Scholar
48. M. H. Rafiei and H. Adeli, A new neural dynamic classification algorithm, IEEE Trans. Neural Netw. Learn. Syst. 28(12) (2017) 3074–3083, https://doi.org/10.1109/TNNLS.2017.2682102. Crossref, Medline, Web of Science, Google Scholar
49. K. M. R. Alam, N. Siddique and H. Adeli, A dynamic ensemble learning algorithm for neural networks, Neural Comput. Appl. 32(10) (2020) 8675–8690. Crossref, Web of Science, Google Scholar
50. D. R. Pereira, M. A. Piteri, A. N. Souza, J. Papa and H. Adeli, FEMa: A finite element machine for fast learning, Neural Comput. Appl. 32(10) (2020) 6393–6404. Crossref, Web of Science, Google Scholar
51. P. Harjule, A. Gurjar, H. Seth and P. Thakur, Text classification on twitter data, in 3rd Int. Conf. Emerging Technologies in Computer Engineering: Machine Learning and Internet of Things (ICETCE-2020), IEEE Conference Record 48199 (2020). Crossref, Google Scholar