Machine LearningNo Access

A Study on the Analysis Model of the Ranking of the Theme of Weibo

Rui Zhang

School of Electrical and Information Engineering, Tianjin University, Tianjin, P. R. China

Department of Software and Communication, Tianjin Sino-German University of Applied Sciences, Tianjin, P. R. China

E-mail Address: hitchcockzhr@163.com

Search for more papers by this author

Zhigang Jin

School of Electrical and Information Engineering, Tianjin University, Tianjin, P. R. China

E-mail Address: zgjin@tju.edu.cn

Corresponding author.

Search for more papers by this author

, and

Xiaohui Liu

The National Computer Network Emergency Response, Technical Team/Coordination Center of China, P. R. China

E-mail Address: xiaohuiliu@163.com

Search for more papers by this author

https://doi.org/10.1142/S0218001418510035Cited by:2 (Source: Crossref)

Abstract

Sina Weibo, the most popular Chinese social platform with hundreds and millions of user-contributed images and texts, is growing rapidly. However, the noise between the image and text, as well as their incomplete correspondence, makes accurate image retrieval and ranking difficult. In this paper, we propose a deep learning framework using visual features, text content and popularity of Weibo to calculate the similarity between the image and the text based on training the model to maximize the likelihood of the target description sentence given the training image. In addition, the retrieval results are reranked using the popularity of the image. The comparison experiment of the large-scale Sina Weibo dataset proves the validity of the proposed method.

Keywords:

References

1. A. Aker and R. Gaizauskas , Generating image descriptions using dependency relational patterns, in Proc. 48th Annual Mettings of the Assoation for Computational Linguistics, (Sweden, 2010), pp. 1250–1258. Google Scholar
2. D. Bahdanau, K. Cho and Y. Bengio, Neural machine translation by jointly learning to align and translate, arXiv:1409.0473. Google Scholar
3. T. Chen, D. Lu, M. Y. Kan et al., Understanding and classifying image tweets, in Proc. 21st ACM Int. Conf. Multimedia, (2013), pp. 781–784. Google Scholar
4. S. W. Cho, M. S. Cha, S. Y. Kim, J. C. Song and K. A. Sohn , Investigating temporal and spatial trends of brand images using twitter opinion mining, in Int. Conf. Information Science and Applications (ICISA), Seoul, 1–4. Google Scholar
5. K. Cho, B. van Merrienboer, C. Gulcehre, F. Bougares, H. Schwenk and Y. Bengio, Learning phrase representations using RNN encoder-decoder for statistical machine translation, arXiv:1406.1078. Google Scholar
6. D. Elliott and F. Keller , Image description using visual dependency representations, in Proc. 2013 Cong. Empirical Methods in Natural Language Processing, (Washington, 2013), pp. 1292–1302. Google Scholar
7. M. Gao, C. Q. Jin, W. N. Qian, X. L. Wang and A. Y. Zhou , Real-time and personalized recommendation on microblogging systems, Chin. J. Comput. 37 (4) (2014) 963–975. Google Scholar
8. J. He, M. Li, H. J. Zhang, H. H. Tong and C. S. Zhang , Generalized manifold ranking-based image retrieval, IEEE Trans. Image Process. 15 (10) (2006) 3170–3177. Crossref, Web of Science, Google Scholar
9. S. Hochreiter and J. Schmidhuber , Long short-term memory, Neural Comput. 9 (8) (1997). Crossref, Web of Science, Google Scholar
10. Y. C. Huang, Q. S. Liu, S. T. Zhang and D. N. Metaxas , Image retrieval via probabilistic hypergraph ranking, in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR) (San Francisco, CA, 2010), pp. 3376–3383. Google Scholar
11. R. Kiros and R. Z. R. Salakhutdinov , Multimodal neural language models, in Proc. NIPS Deep Learning Workshop, 2013. Google Scholar
12. R. Kiros, R. Salakhutdinov and R. S. Zemel , Unifying visual semantic embeddings with multimodal neural language models, Trans. Assoc. Computational Linguistics (2015). Google Scholar
13. P. Kuznetsova, V. Ordonez, A. C. Berg, T. L. Berg and Y. Choi , Collective generation of natural image descriptions, in Proc. 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1 (ACL ’12), Vol. 1, Association for Computational Linguistics, Stroudsburg, PA, USA (2012), pp. 359–368. Google Scholar
14. P. Kuznetsova, V. Ordonez, T. Berg and Y. Choi , Treetalk: Composition and compression of trees for image descriptions, Trans, Assoc. Comput. Linguistics, Vol. 2 (2014) pp. 351–362. Google Scholar
15. Y. A. LeCun, L. Bottou, G. B. Orr and K.-R. Muller , Efficient Backprop, in Neural Networks: Tricks of the Trade (Springer, Heidelberg, 2012), pp. 9–48. Crossref, Google Scholar
16. P. Li, M. Wang, J. Cheng, C. Xu and H. Lu , Spectral hashing with semantically consistent graph for image indexing, IEEE Trans. Multimed. 15 (1) (2013) 141–152. Crossref, Web of Science, Google Scholar
17. D. Liu, X. S. Hua, L. Yang, M. Wang and H. J. Zhang , Tag ranking, in Proc. ACM Conf. World Wide Web (Madrid, 2009), pp. 351–360. Google Scholar
18. K. Liu, L. M. Zhang, Y. W. Sun and X. Y. Lin , An automatic image annotation algorithm using deep boltzmann machine and canonical correlation analysis, J. Xian Jiaotong Univ. 49 (6) (2015) 33–38. Google Scholar
19. J. Mao, W. Xu, Y. Yang, J. Wang and A. Yuille, Explain images with multimodal recurrent neural networks, in arXiv:1410.1090. Google Scholar
20. J. Mao, W. Xu, Y. Yang, J. Wang and A. Yuille, Deep captioning with multimodal recurrent neural networks (m-rnn), arXiv:1412.6632. Google Scholar
21. M. Mitchell, X. Han, J. Dodge, A. Mensch, A. Goyal, A. Berg, K. Yamaguchi, T. Berg, K. Stratos and H. Daumé III , Midge: Generating image descriptions from computer vision detections, in Proc. 13th Conf. of the European Chapter of the Association for Computation Linguistics, (EACL ’12), Association for Computational Linguistics, Stroudsburg, PA, USA (2012), pp. 747–756. Google Scholar
22. R. Ren, C. K. Pang, L. Ma et al., An experimental study of correlation between text and image similarity by information fusion approach, Proc. 18th Asia Pacific Symp. Intelligent and Evolutionary Systems, (Springer International Publishing, 2015), pp. 671–685. Google Scholar
23. I. Sutskever, O. Vinyals and Q. V. Le, Sequence to sequence learning with neural networks, arXiv:1409.3215. Google Scholar
24. J. L. Tian and W. Zhao , Words similarity algorithm based on TongyiciCilin in semantic web adaptive learning system, J. Jilin Univ. 28 (6) (2010) 602–608. Google Scholar
25. Y. Jie, L. Maofu, L. Ya and H. Huijun , Combination of multi-modal features for Weibo image and text correlation recognition, Appl. Res. Comput. 34 (9) (2017) 1–7. Google Scholar
26. Y. Zhang, M. Chen, S. Mao, L. Hu and V. Leung , CAP: Community activity prediction based on big data analysis, IEEE Netw. 28 (4) (2014) 52–57. Crossref, Web of Science, Google Scholar