Question: Hello I need to write a program in python, which will read a data file and will output the minimum amount spend by a customer,

Hello I need to write a program in python, which will read a data file and will output the minimum amount spend by a customer,

The code I have been working on is this, but I am getting errors can you please help me to fix it?

CODE

from pyspark import SparkConf, SparkContext

conf = SparkConf().setMaster("local").setAppName("Min TotalSpends") sc = SparkContext(conf = conf)

def parseLine(line): fields = line.split(',') CustomerID = fields[0] ItemID = fields[2] TotalSpent = fields[3] return (CustomerID, ItemID, TotalSpent)

lines = sc.textFile("file:///SparkCourse/customer-orders.csv") parsedLines = lines.map(parseLine) minSpent = parsedLines.filter(lambda x: "TMIN" in x[1]) stationT = minSpent.map(lambda x: (x[0], x[2])) minSpent = stationT.reduceByKey(lambda x, y: min(x,y)) results = minSpent.collect();

for result in results: print(result[0] + "\t{:.2f}F".format(result[1]))

# I am providing the data file bellow (the first column is the customer id, the second column shows item id and the third column exhibits total amount spend by a customer) Data file is called customer-orders.csv is the following

CustomerID ItemID TotalSpent
35 5368 65.89
2 3391 40.64
47 6694 14.98
29 680 13.08
91 8900 24.59
70 3959 68.68
85 1733 28.53
53 9900 83.55
14 1505 4.32
51 3378 19.8

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!