Special IssueNo Access

A PROBABILISTIC APPROACH TO MULTI-DOCUMENT SUMMARIZATION FOR GENERATING A TILED SUMMARY

Department of Computer Science & Engineering, IIT Madras, Chennai, Tamil Nadu 600 024, India

Working as faculty in Department of Information Technology, Jerusalem College of Engineering, Chennai-601 302, Tamil Nadu, India.

Search for more papers by this author

S. RAMAN

Department of Computer Science & Engineering, IIT Madras, Chennai, Tamil Nadu 600 024, India

Search for more papers by this author

, and

B. RAVINDRAN

Department of Computer Science & Engineering, IIT Madras, Chennai, Tamil Nadu 600 024, India

Search for more papers by this author

https://doi.org/10.1142/S1469026806001976Cited by:3 (Source: Crossref)

Abstract

Data availability is not a major issue at present times in view of the widespread use of Internet; however, information and knowledge availability are the issues. Due to data overload and time-critical nature of information need, automatic summarization of documents plays a significant role in information retrieval and text data mining. This paper discusses the design of a multi-document summarizer that uses Katz's K-mixture model for term distribution. The model helps in ranking the sentences by a modified term weight assignment. Highly ranked sentences are selected for the final summary. The sentences that are repetitive in nature are eliminated, and a tiled summary is produced. Our method avoids redundancy and produces a readable (even browsable) summary, which we refer to as an event-specific tiled summary. The system has been evaluated against the frequently occurring sentences in the summaries generated by a set of human subjects. Our system outperforms other auto-summarizers at different extraction levels of summarization with respect to the ideal summary, and is close to the ideal summary at 40% extraction level.

Keywords:

Remember to check out the Most Cited Articles!
Check out these titles in artificial intelligence!

References

M. Berryet al., SIAM Rev. 37, 573 (1995), DOI: 10.1137/1037127. Crossref, Google Scholar
R. Barzilay and M. Elhadad, Using lexical chains for text summarization, Proc. ACL Workshop on Intelligent Scalable Text Summarization (1997) pp. 10–17. Google Scholar
Julian M. Kupiec, J. Pedersen and F. Chen, A trainable document summarizer, Proc. 18th ACM-SIGIR Conf. on Research and Development in Information Retrieval (1995) pp. 68–73, DOI: 10.1145/215206.215333. Google Scholar
M. Hajime and O. Manabu, A comparison of summarization methods based on task-based evaluation, 2nd International Conference on Language Resources and Evaluation, LREC-2000 (2000) pp. 633–639. Google Scholar
A. Tombros and M. Sanderson, Advantages of query biased summaries in information retrieval, Proc. SIGIR-98 (1998) pp. 2–10, DOI: 10.1145/290941.290947. Google Scholar
H. P. Edmundson, J. ACM. 16(2), 264 (1969), DOI: 10.1145/321510.321519. Crossref, Google Scholar
C. D. Paice, Inf. Process. Management 26(1), 264 (1990), DOI: 10.1016/0306-4573(90)90014-S. Google Scholar
C. D. Manning and H. Schutze , Foundations of Statistical Natural Language Processing ( The MIT Press , London, England , 2001 ) . Google Scholar
H. P. Luhn, IBM J. Res. Dev. 2(2), 159 (1958). Crossref, Google Scholar
G. Salton and C. Buckley, Inf. Process. Management 24, 513 (1988), DOI: 10.1016/0306-4573(88)90021-0. Crossref, Google Scholar
S. Harter, A probabilistic approach to automatic keyword indexing, PhD dissertation, University of Chicago (1975) . Google Scholar
D. R. Radev, H. Jing and M. Budzikowska, Centroid-based summarization of multiple document: Sentence extraction, utility-based evaluation, and user studies, ANLP-NAACL Workshop on Summarization (2000) pp. 21–29. Google Scholar
Neto and Santos, Document clustering and text summarization, Proc. 4th Int. Conf. on Practical Applications of Knowledge Discovery and Data Mining, PADD-2000 (2000) pp. 41–55. Google Scholar
C. Y. Lin and E. H. Hovy, The automated acquisition of topic signatures for text summarization, Proc. COLING Conf. (2000) pp. 595–601, DOI: 10.3115/990820.990892. Google Scholar
M. A. Hearst and C. Plaunt, Subtopic structure for full-length document access, Proc. 16th Ann. Int. ACM SIGIR Conference on Research and Development in Information Retrieval (1993) pp. 59–68. Google Scholar
N. L. Johnson and S. Katz , Discrete Distributions ( The Houghton Mifflin Press , MA , 1969 ) . Google Scholar
S. M. Katz, Natural Language Eng. 2(1), 15 (1995), DOI: 10.1017/S1351324996001246. Crossref, Google Scholar
K. W. Church and W. Gale, Natural Language Eng. 1(2), 163 (1995). Crossref, Google Scholar
M. Saravanan and S. Raman, The term distribution model for summarization of multiple documents, Proc. Indo-European Conf. on Multilingual Communication Technologies (IEMCT) (2002) pp. 182–192. Google Scholar
H. Jinget al., Summarization evaluation methods: Experiments and analysis, Proc. AAAI98 Spring Symp. Intell. Text Summarization (1998) pp. 60–68. Google Scholar
http://iil.cs.iitm.ernet.in/publications/dataset . Google Scholar
http://www.sinope.nl/en/sinope/home.html . Google Scholar
http://www.copernic.com/products/summarizer . Google Scholar
M. Saravananet al., Improved Porter's algorithm for root word stemming, Proc. Int. Conf. on Natural Language Processing (ICON'02) (2002) pp. 21–30. Google Scholar