(Data Mining short Python code only)

I just need some code to start bellow. Please help me write some Python code. Only fill the YOUR CODE HERE part .

I already asked step 1 and step 2 before. you could answer only step 3 part.

Frequent Pattern Mining

implementing the APRIORI algorithm for Frequent Pattern Mining using Python. In order to implement the full algorithm, you will need to write two helper functions: a candidate generator and a support counter. Once you have a working candidate generator and support counter, you can implement the APRIORI algorithm and run it on the three provided datasets (described below). Each function has type hints to help you understand what the expected inputs/outputs are.

As in Assignment 2, this notebook povides a template of the functions you will need to implement. Your task is to read and understand each step of this assignment and to fill in the areas marked YOUR CODE HERE. DO NOT alter any code outside of the marked areas or your work may not run or generate the correct output.

Data

3 files each including a transaction dataset; each row is a seperate transaction so transaction ID's are omitted.

dataset1.txt -> 100 transactions

dataset2.txt -> 400 transactions

dataset3.txt -> 800 transactions

Your code shoule be able to handle all 3 datasets in a reasonable timeframe (<1min each at most)

##Python code###

# Imports - DO NOT CHANGE

from itertools import combinations

from time import time

from typing import List, Dict

from gcsfs import GCSFileSystem

# Download datasets - may take a few seconds

fs = GCSFileSystem(project='csci4800-dm', token='anon', access='read_only')

fs.get('csci4800-data/assignment_3/dataset1.txt', './dataset1.txt')

fs.get('csci4800-data/assignment_3/dataset2.txt', './dataset2.txt')

fs.get('csci4800-data/assignment_3/dataset3.txt', './dataset3.txt')

# Declare timed decorator for timing functions

"""

NOTE: This is a helper function (called a decorator) that outputs the runtime of a function, it is used by placing

@timed above the declaration of a function.

"""

def timed(f):

def time_wrap(*args, **kwargs):

t_start = time()

result = f(*args, **kwargs)

t_end = time()

if f.__name__ == 'gen_candidates':

print("func: {} took: {:E} sec and generated {} candidates for k = {}".format(

f.__name__, t_end-t_start, len(result), args[1]))

else:

print("func: {} took: {:E} sec".format(f.__name__, t_end-t_start))

return result

return time_wrap

Step 1 - Candidate Generator

Write the function that generates potential frequent candidates at each level of the iterative APRIORI algorithm. Use the lecture notes for a description of how the algorithm works, if you need specific implementation help: first read the python documentation, if that does not solve the issue then you may stop by for the TA's office hours.

Notes:

Even though you are not using SQL, the SQL implementation in the lecture notes (slide 15) provides a good blueprint for designing this function using Python

A good way to store the candidates is as a dictionary (Python dict) with key -> values pairs being candidate -> count setting the initial counts all to 0.

Python dicts cannot store sets or lists as keys, but you can convert a string to a set/list and vice-versa with ' '.join() and .split(' ')

You can iterate through dict keys using a for statement (e.g. for itemset in L.keys())

The Python itertools module provides a function (combinations) for generating all subsets of length k of a set

Code gose here:

@timed

def gen_candidates(L: Dict[str, int], k: int) -> Dict[str, int]:

"""

Generates candidate itemsets from the k-1 frequent itemsets

:param L: frequent itemsets at k-1 as a dict of itemset -> count pairs

:param k: length of itemsets being generated

:returns: dict of candidate itemsets

"""

C = {}

"""

*

* YOUR CODE HERE

*

"""

Step 2 - Support Counter

Write the function that counts the support for each candidates generated by your algorithm in Step 1.

Notes:

This function only needs to generate the counts, it does not handle pruning

Python has functions all() and any() that check if all or any elements of a list are true respectively

As with before the itemsets are whitespace delimited strings that should be split to form arrays

Code goes here:

@timed

def count_support(candidates: Dict[str, int], transactions: List[str]) -> Dict[str, int]:

"""

Counts support for each candidate itemset

:param candidates: candidate itemsets generated using the candidate generator

:param transactions: list of transactions, delimited by whitespaces

:returns: a dict of candidate -> support pairs

"""

supports = candidates.copy()

"""

*

* YOUR CODE HERE

*

"""

Step 3 - Main Loop

Write the main loop that runs the APRIORI algorithm using the Candidate Generator and Support Counter implemented in steps 1 and 2.

Notes:

You may return a dict of itemset -> count pairs rather than just the sets of frequent items

A useful way to generate L_k is by using dict comprehension (see docs for dict above) on the itemset -> count pairs

The first set of candidates has been generated for you (itemsets of length 1, i.e. k = 1)

@timed

def APRIORI(transactions: List[str], min_support: int) -> List[set]:

"""

Runs the APRIORI algorithm on a set of transactions

:param transactions: list of transactions, delimited by whitespace

:param min_support: minimum support

:return: list of frequent itemsets for each value of k

"""

# Generate the first set of candidates

C0 = {}

for line in transactions:

for t in line.rstrip().split(' '):

C0[t] = 0

# Count support for each candidate

C0 = count_support(C0, transactions)

# Generate the first frequent itemset (k = 1)

L_k = { c:count for c, count in C0.items() if count > min_support }

# Add the first frequent itemset to the union for k = 1

L_union = [L_k]

# Begin at k = 2

k = 2

while len(L_k) > 0:

"""

*

* YOUR CODE HERE

*

"""

return L_union