No Access

WEB MINING: A SURVEY OF CURRENT RESEARCH, TECHNIQUES, AND SOFTWARE

QINGYU ZHANG

Department of Computer & Information Technology, Arkansas State University, State University, Arkansas 72467-0130, USA

Corresponding author.

Search for more papers by this author

and

RICHARD S. SEGALL

Department of Computer & Information Technology, Arkansas State University, State University, Arkansas 72467-0130, USA

Search for more papers by this author

https://doi.org/10.1142/S0219622008003150Cited by:56 (Source: Crossref)

Abstract

The purpose of this paper is to provide a more current evaluation and update of web mining research and techniques available. Current advances in each of the three different types of web mining are reviewed in the categories of web content mining, web usage mining, and web structure mining. For each tabulated research work, we examine such key issues as web mining process, methods/techniques, applications, data sources, and software used. Unlike previous investigators, we divide web mining processes into the following five subtasks: (1) resource finding and retrieving, (2) information selection and preprocessing, (3) patterns analysis and recognition, (4) validation and interpretation, and (5) visualization. This paper also reports the comparisons and summaries of selected software for web mining. The web mining software selected for discussion and comparison in this paper are SPSS Clementine, Megaputer PolyAnalyst, ClickTracks by web analytics, and QL2 by QL2 Software Inc. Applications of these selected web mining software to available data sets are discussed together with abundant presentations of screen shots, as well as conclusions and future directions of the research.

Keywords:

References

L. Abraham and V. Ramos, Web usage mining using artificial ant colony clustering and genetic programming, CEC'03 — Congress on Evolutionary Computation (IEEE Press, Canberra, Australia, 2003) pp. 1384–1391. Google Scholar
N. Barsagade, Web usage mining and pattern discovery: A survey paper, Computer Science and Engineering Dept., CSE Tech Report 8331 (Southern Methodist University, Dallas, Texas, USA, 2003) . Google Scholar
S. Chakrabarti , Mining the Web: Discovering Knowledge from Hypertext Data ( Elsevier Science & Technology Books , 2003 ) . Google Scholar
R. Chau, C. Yeh and K. Smith, Personalized multilingual web content mining, KES (2004) pp. 155–163. Google Scholar
L. Chen, W. Lian and W. Chue, Inform. Process. Management: Int. J. 41(5), 1225 (2005). Web of Science, Google Scholar
T. S. Clendaniel , The Data Administration Newsletter ( R.S. Seiner, Publisher , 2002 ) , http://www.tdan.com/i019fe02.htm . Google Scholar
R. Cooley, ACM Trans. Internet Tech. 3(2), 93 (2003), DOI: 10.1145/767193.767194. Google Scholar
R. Cooley , B. Mobasher and J. Srivastava , Web mining: Information and pattern discovery on the world wide web , ICTAI ( 1997 ) , DOI: 10.1109/TAI.1997.632303 . Google Scholar
R. Cooley, B. Mobasher and J. Srivastava, Data preparation for mining worldwide browsing patterns (2003), http://maya.cs.depaul.edu/~classes/ect584/papers/cms-kais.pdf . Google Scholar
S. Cyrus and F. Banaei-Kashani, INFORMS J. Comput. Special Issue on Data Mining 15(2), 123 (2003). Web of Science, Google Scholar
J. Darmont, O. Boussaid and F. Bentayeb, Warehousing Web Data (2007), http://www.arxiv.org/ftp/arxiv/papers/0705/0705.1456.pdf . Google Scholar
O. Etzioni, Commun. ACM 39(11), 65 (1996), DOI: 10.1145/240455.240473. Web of Science, Google Scholar
X. Fang and O. Sheng, ACM Trans. Internet Tech. 4(2), 209 (2004), DOI: 10.1145/990301.990306. Google Scholar
K. Fenstermacher and M. Ginsburg, J. Am. Soc. Inform. Sci. Tech. 54(7), 625 (2003), DOI: 10.1002/asi.10255. Web of Science, Google Scholar
J. Furnkranz, ÖGAI-J. 21(2), 17 (2002). Web of Science, Google Scholar
S. Graves et al. , Deployable suite of data mining web services for online science data repositories , 23rd Conf. IIPS, 87th American Meteorological Society Annual Meeting . Google Scholar
D. Greening, Data mining on the web: There's gold in that mountain of data (2006) http://www.webtechniques.com/archives/2000/01/greening/ . Google Scholar
S. Guan and P. McMullen, Int. J. Inform. Technol. Decision Making 4(1), 97 (2005), DOI: 10.1142/S0219622005001404. Link, Web of Science, Google Scholar
J. Han and C. Chang, Computer 54 (2002), http://www-faculty.cs.uiuc.edu/~hanj/pdf/computer02.pdf. Google Scholar
B. Hay, G. Wets and K. Vanhoof, Knowledge Inform. Syst. 6(2), 150 (2004). Web of Science, Google Scholar
A. Joshi, Web mining (2001), http://www.cs.umbc.edu/~ajoshi/web_mine . Google Scholar
A. Joshi, Web/data mining and personalization, University of Maryland Baltimore County (UMBC) eBiquity Research Area (2001), http://ebiquity.umbc.edu/project/html/id/17/Web-Data-Mining-and-Personalization . Google Scholar
D. Kanellopoulos and S. Kotsiantis, Int. Rev. Comput. Software 3(1), 428 (2001). Google Scholar
P. Kolari and A. Joshi, Comput. Sci. Eng. 42 (2004), DOI: 10.1109/MCSE.2004.23. Google Scholar
R. Kosala and H. Blockeel, ACM SIGKDD Explor. 2, 1 (2000), DOI: 10.1145/360402.360406. Google Scholar
K. Lauet al., J. Database Marketing Customer Strategy Management 12(1), 32 (2004), DOI: 10.1057/palgrave.dbm.3240241. Google Scholar
J. W. Liang, Introduction to text and web mining, Seminar at North Carolina Technical University (2003), http://www.database.cis.nctu.edu.tw/seminars/2003F/TWM/slides/p.ppt . Google Scholar
B. Liu, Web content mining (2005), http://www.cs.uic.edu/~liub/WebContent-Mining.html . Google Scholar
C. Liu, Web content mining (29 November 2006), CM SIGKDD Webcast . Google Scholar
B. Liu , Web Data Mining: Exploring Hyperlinks, Contents and Usage Data ( Springer Verlag Press , 2007 ) . Google Scholar
B. Liu and K. Chang, SIGKDD Explorations 6(2), 1 (2004), DOI: 10.1145/1046456.1046457. Google Scholar
Z. Markov and D. T. Larose , New Report "Data Mining the Web: Uncovering Patterns in Web Content, Structure, and Usage" Features Web Structure Mining, Web Content Mining and Web Usage Mining ( John Wiley and Sons , 2007 ) , http://biz.yahoo.com/bw/070525/20070525005213.html?.v=1 . Google Scholar
Megaputer Intellegence Inc., WebAnalyst architecture (2007), http://www.megputer.com/products/wa/architechture.php3 . Google Scholar
B. Mobasher, Web data mining for business intelligence, ECT 584, DePaul University, Chicago, IL (2007), http://maya.cs.depaul.edu/~classes/ect584/papers/mobasher.pdf . Google Scholar
B. Mobasher, R. Colley and J. Srivastava, Commun. ACM 43(8), 142 (2000), DOI: 10.1145/345124.345169. Web of Science, Google Scholar
B. Mobasher et al. , Integrating web usage and content mining for more effective personalization , EC-Web ( 2000 ) . Google Scholar
Z. Pabarskaite and A. Raudys, J. Intell. Inform. Syst. 28(1), 79 (2007), DOI: 10.1007/s10844-006-0004-1. Web of Science, Google Scholar
S. Palmer, The semantic web: An introduction (2001), http://infomesh.net/2001/swintro/ . Google Scholar
D. Pierrakoset al., User Model. User-Adapt. Interact. 13(4), 311 (2003), DOI: 10.1023/A:1026238916441. Web of Science, Google Scholar
QL2 software (2007), http://www.ql2.com, viewed 5 June 2007 . Google Scholar
A. Scime, J. Intell. Inform. Syst. 22(3), 211 (2004), DOI: 10.1023/B:JIIS.0000019288.63141.e4. Web of Science, Google Scholar
A. Scime , Web Mining: Applications and Techniques ( Idea Group Publishing , Hershey, PA , 2005 ) . Google Scholar
Semantic Web Agreement Group, What is the semantic web? (2001) http://swag.webns.net/WhatIsSW . Google Scholar
K. A. Smith and A. Ng, Decision Support Syst. 35(2), 245 (2003), DOI: 10.1016/S0167-9236(02)00109-4. Web of Science, Google Scholar
Q. Song and M. Shepperd, Comput. Indus. 57(7), 622 (2006), DOI: 10.1016/j.compind.2005.11.006. Web of Science, Google Scholar
M. Spiliopoulou, Commun. ACM 43(8), 127 (2000), DOI: 10.1145/345124.345167. Web of Science, Google Scholar
SPSS, Web mining for Clementine (2007) http://www.spss.com/web_mining_for_clementine, viewed 16 May 2007 . Google Scholar
J. Srivastavaet al., SIGKDD Explor. 1(2), 12 (2000), DOI: 10.1145/846183.846188. Google Scholar
W3Schools, Semantic web tutorial (2008) http://www.w3schools.com/semweb/default.asp . Google Scholar
Web Analytics (2007) http://www.clicktracks.com/, viewed as 25 October 2007 . Google Scholar
Wikipedia, Web mining (2007) http://en.wikipedia.org/wiki/Web_mining . Google Scholar
Wikipedia, Semantic web (2008) http://en.wikipedia.org/wiki/Semantic_Web . Google Scholar
Q. Yang and X. Wu, Int. J Inform. Technol. Decision Making 5(4), 597 (2006), DOI: 10.1142/S0219622006002258. Link, Web of Science, Google Scholar
A. Zanasi et al. , Data mining VII: Data, text and web mining and their business applications (Data Mining and Information Engineering 2006) , Transactions on Information and Communication Technologies 37 ( 2006 ) . Google Scholar
H. Zhanget al., World Wide Web 6(2), 131 (2003), DOI: 10.1023/A:1023618504691. Web of Science, Google Scholar
N. Zhong, Int. J. Inform. Technol. Decision Making 5(4), 713 (2006), DOI: 10.1142/S0219622006002283. Link, Web of Science, Google Scholar
G. Adomavicius and A. Tuzhilin, Data Mining Knowledge Discov. 5, 33 (2001), DOI: 10.1023/A:1009839827683. Web of Science, Google Scholar
R. Amarasiri and D. Alahakoon, Neural Comput. Appl. 13, 149 (2004). Web of Science, Google Scholar
R. Boncella, Commun. Assoc. Inform. Syst. 12, 327 (2005). Web of Science, Google Scholar
M. Ceci and D. Malerba, J. Intell. Inform. Syst. 28, 37 (2007), DOI: 10.1007/s10844-006-0003-2. Web of Science, Google Scholar
M. Chauet al., Decision Support Syst. 35, 167 (2003), DOI: 10.1016/S0167-9236(02)00103-3. Web of Science, Google Scholar
R. Chau and C. Yeh, Neural Comput. Appl. 13, 140 (2004). Web of Science, Google Scholar
Z. Chen, A. Fu and F. Tong, World Wide Web 6, 259 (2003), DOI: 10.1023/A:1024606901978. Web of Science, Google Scholar
R. Chen, K. Sivakumar and H. Kargupta, Knowledge Inform. Syst. 6, 164 (2004). Web of Science, Google Scholar
S. Dustdar and R. Gombotz, Int. J. Business Process Integrat. Management 1, 256 (2007), DOI: 10.1504/IJBPIM.2006.012624. Google Scholar
M. Eirinaki and M. Vazirgiannis, ACM Trans. Internet Tech. 3, 1 (2003), DOI: 10.1145/643477.643478. Web of Science, Google Scholar
B. Ezeife and Y. Lu, Data Mining Knowledge Discov. 10, 5 (2005), DOI: 10.1007/s10618-005-0248-3. Web of Science, Google Scholar
F. Faccaand and P. Lanzi, Data Knowledge Eng. 53, 225 (2005). Web of Science, Google Scholar
S. Flescaet al., World Wide Web 8, 317 (2005), DOI: 10.1007/s11280-005-1315-9. Web of Science, Google Scholar
P. Giudici and R. Castelo, Data Mining Knowledge Discov. 5, 183 (2001), DOI: 10.1023/A:1011469000311. Web of Science, Google Scholar
B. Gregg and S. Walczak, Commun. ACM 49, 78 (2006), DOI: 10.1145/1125944.1125945. Web of Science, Google Scholar
W. Grossmann, M. Hudec and R. Kurzawa, Int. J. Electron. Business 2, 480 (2004), DOI: 10.1504/IJEB.2004.005881. Web of Science, Google Scholar
H. Han and R. Elmasri, J. Intell. Inform. Syst. 22, 237 (2004), DOI: 10.1023/B:JIIS.0000019278.84222.b7. Web of Science, Google Scholar
B. Haruechaiyasak and M. Shyu, Int. J. Comput. Appl. Tech. 27, 298 (2007), DOI: 10.1504/IJCAT.2006.012000. Web of Science, Google Scholar
B. Huang and T. Chou, Electron. Commerce Res. Appl. 3, 266 (2004), DOI: 10.1016/j.elerap.2004.04.002. Google Scholar
X. Huanget al., J. Am. Soc. Inform. Sci. Tech. 56, 1290 (2004). Google Scholar
X. Jiang, Inform. Software Tech. 46, 55 (2004). Web of Science, Google Scholar
K. Joshi, A. Joshi and Y. Yesha, Distributed Parallel Databases 13, 161 (2003), DOI: 10.1023/A:1021515408295. Web of Science, Google Scholar
Kdnuggets, Software: Web Mining and Web Usage Mining, http://www.kdnuggets.com/software/web-mining.html . Google Scholar
D. Kim, H. Jung and L. Geunbae, Inform. Process. Manag. 39, 623 (2003), DOI: 10.1016/S0306-4573(03)00004-9. Web of Science, Google Scholar
Y. Kotb, K. Gondow and T. Katayama, J. Int. Inform. Syst. 22, 257 (2004), DOI: 10.1023/B:JIIS.0000019279.81927.c5. Web of Science, Google Scholar
Y. Kuo and L. Chen, Expert Syst. Appl. 21, 203 (2001), DOI: 10.1016/S0957-4174(01)00040-9. Web of Science, Google Scholar
C. Lee, Y. Kim and P. Rhee, Expert Syst. Appl. 21, 131 (2001), DOI: 10.1016/S0957-4174(01)00034-3. Web of Science, Google Scholar
A. Maguitmanet al., World Wide Web 9, 431 (2006), DOI: 10.1007/s11280-006-8562-2. Web of Science, Google Scholar
A. Nanopoulos and Y. Manolopoulos, Data Knowledge Eng. 37, 243 (2001), DOI: 10.1016/S0169-023X(01)00008-8. Web of Science, Google Scholar
Z. Pabatsjaute, Intll. Data Anal. 7, 141 (2003). Web of Science, Google Scholar
D. Roussinov and F. Zhao, J. Enterprise Inform. Management 17, 424 (2004), DOI: 10.1108/17410390410566715. Google Scholar
B. Sakkopoulos, D. Kanellopoulos and A. Tsakalidis, Int. J. Metadata Semant. Ontol. 1, 66 (2006), DOI: 10.1504/IJMSO.2006.008771. Google Scholar
M. Shyu, C. Haruechaiyasak and S. Chen, Knowledge Inform. Syst. 10, 515 (2006), DOI: 10.1007/s10115-006-0004-z. Web of Science, Google Scholar
K. Smith and A. Ng, Decision Support Syst. 35, 245 (2003), DOI: 10.1016/S0167-9236(02)00109-4. Web of Science, Google Scholar
P. Smyth, D. Pregibon and C. Faloutsos, Commun. ACM 45, 33 (2002). Google Scholar
A. Thuraisingham , Web Data Mining and Applications in Business Intelligence and Counter-Terrorism ( CRC Press , 2004 ) . Google Scholar
L. van Wel and R. Lambèr, Ethics Inform. Tech. 6, 129 (2004), DOI: 10.1023/B:ETIN.0000047476.05912.3d. Web of Science, Google Scholar
Y. Wu and A. Chen, World Wide Web 5, 67 (2002), DOI: 10.1023/A:1015750423727. Google Scholar
Q. Yang, T. Li and K. Wang, Data Mining Knowledge Discov. 8, 253 (2004), DOI: 10.1023/B:DAMI.0000023675.04946.f1. Web of Science, Google Scholar
Q. Yang, T. Li and K. Wang, Appl. Artific. Intell. 17, 431 (2003), DOI: 10.1080/713827182. Web of Science, Google Scholar
Q. Yang and H. Zhang, World Wide Web 4, 299 (2001), DOI: 10.1023/A:1015185802583. Google Scholar
A. Zhang and Y. Dong, Comput. Networks 39, 303 (2002), DOI: 10.1016/S1389-1286(02)00211-6. Web of Science, Google Scholar