Question: Considering running the PCY algorithm on a dataset with 5 0 0 million baskets to count frequent pairs of items. Suppose each basket contains n

Considering running the PCY algorithm on a dataset with 500 million baskets to count frequent pairs of items. Suppose each basket contains n items and there are d distinct item-pairs amongst all of the baskets. Consider the following setup during the first pass of PCY: Besides keeping the counters for every singleton itemset observed during the first pass, we can still afford to store in main memory 300 million integers, each of which is a bucket. Assume further that d is much larger than the total number of buckets available, i.e., d >>300 million. (a)[10 marks] As a function of n and/or d, what is the minimum support threshold s we can allow if the average count for a bucket should be no more than 40% of the threshold? (b)[10 marks] Suppose that A, B, C, D, E, and F are all the items under consideration. For a particular support threshold, the maximal frequent itemsets are {A, B, D} and {C, E}. What are all the other frequent itemsets?
Considering running the PCY algorithm on a dataset with 500 million baskets to count frequent pairs of items. Suppose each basket contains n items and there are d distinct item-pairs amongst all of the baskets. Consider the following setup during the first pass of PCY: Besides keeping the counters for every singleton itemset observed during the first pass, we can still afford to store in main memory 300 million integers, each of which is a bucket. Assume further that d is much larger than the total number of buckets available, i.e., d >>300 million. (a)[10 marks] As a function of n and/or d, what is the minimum support threshold s we can allow if the average count for a bucket should be no more than 40% of the threshold? (b)[10 marks] Suppose that A, B, C, D, E, and F are all the items under consideration. For a particular support threshold, the maximal frequent itemsets are {A, B, D} and {C, E}. What are all the other frequent itemsets?

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Programming Questions!