Distributed and scalable sequential pattern mining through stream processing

Chun Chieh Chen*, Hong-Han Shuai, Ming Syan Chen

*Corresponding author for this work

Research output: Contribution to journalArticle

12 Scopus citations

Abstract

Scalability is a primary issue in existing sequential pattern mining algorithms for dealing with a large amount of data. Previous work, namely sequential pattern mining on the cloud (SPAMC), has already addressed the scalability problem. It supports the MapReduce cloud computing architecture for mining frequent sequential patterns on large datasets. However, this existing algorithm does not address the iterative mining problem, which is the problem that reloading data incur additional costs. Furthermore, it did not study the load balancing problem. To remedy these problems, we devised a powerful sequential pattern mining algorithm, the sequential pattern mining in the cloud-uniform distributed lexical sequence tree algorithm (SPAMC-UDLT), exploiting MapReduce and streaming processes. SPAMC-UDLT dramatically improves overall performance without launching multiple MapReduce rounds and provides perfect load balancing across machines in the cloud. The results show that SPAMC-UDLT can significantly reduce execution time, achieves extremely high scalability, and provides much better load balancing than existing algorithms in the cloud.

Original languageEnglish
Pages (from-to)365-390
Number of pages26
JournalKnowledge and Information Systems
Volume53
Issue number2
DOIs
StatePublished - 1 Nov 2017

Keywords

  • Big data
  • Cloud computing
  • Data mining
  • MapReduce
  • Sequential pattern mining
  • Streaming MapReduce

Fingerprint Dive into the research topics of 'Distributed and scalable sequential pattern mining through stream processing'. Together they form a unique fingerprint.

  • Cite this