Question: Hello I need to write a program in python, which will read a data file and will output the minimum amount spend by a customer,
Hello I need to write a program in python, which will read a data file and will output the minimum amount spend by a customer,
The code I have been working on is this, but I am getting errors can you please help me to fix it?
CODE
from pyspark import SparkConf, SparkContext
conf = SparkConf().setMaster("local").setAppName("Min TotalSpends") sc = SparkContext(conf = conf)
def parseLine(line): fields = line.split(',') CustomerID = fields[0] ItemID = fields[2] TotalSpent = fields[3] return (CustomerID, ItemID, TotalSpent)
lines = sc.textFile("file:///SparkCourse/customer-orders.csv") parsedLines = lines.map(parseLine) minSpent = parsedLines.filter(lambda x: "TMIN" in x[1]) stationT = minSpent.map(lambda x: (x[0], x[2])) minSpent = stationT.reduceByKey(lambda x, y: min(x,y)) results = minSpent.collect();
for result in results: print(result[0] + "\t{:.2f}F".format(result[1]))
# I am providing the data file bellow (the first column is the customer id, the second column shows item id and the third column exhibits total amount spend by a customer) Data file is called customer-orders.csv is the following
| CustomerID | ItemID | TotalSpent |
| 35 | 5368 | 65.89 |
| 2 | 3391 | 40.64 |
| 47 | 6694 | 14.98 |
| 29 | 680 | 13.08 |
| 91 | 8900 | 24.59 |
| 70 | 3959 | 68.68 |
| 85 | 1733 | 28.53 |
| 53 | 9900 | 83.55 |
| 14 | 1505 | 4.32 |
| 51 | 3378 | 19.8 |
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
