Frequent itemset mining for big data
Faculty of Sciences. Mathematics and Computer Science
New York, N.Y. :IEEE, 2013
IEEE Big Data 2013 : International Conference on Big Data, October 6-9, 2013, Santa Clara, Calif., USA
University of Antwerp
Frequent Itemset Mining (FIM) is one of the most well known techniques to extract knowledge from data. The combinatorial explosion of FIM methods become even more problematic when they are applied to Big Data. Fortunately, recent improvements in the field of parallel programming already provide good tools to tackle this problem. However, these tools come with their own technical challenges, e.g. balanced data distribution and inter-communication costs. In this paper, we investigate the applicability of FIM techniques on the MapReduce platform. We introduce two new methods for mining large datasets: Dist-Eclat focuses on speed while BigFIM is optimized to run on really large datasets. In our experiments we show the scalability of our methods.