Subject - Computer Science (Big Data Engineering)

Note: Assignment last date 30-Jan. Early response appreciated. Will upvote all answer.

Consider the following employee dataset present in the file emp.csv. Attempt the following question related to Spark data frames using this dataset. [2 + 2 +1 +2 +3 = 10) Temployee_name| department|state| salary|age|bonus | James Sales Michael Sales Robert| Sales Maria Finance Raman Finance | Scott| Finance Jen Finance Jeff Marketing | Kumar| Marketing | NY| 90000| 34|10000| NY 86000| 5620000|| CA| 81000| 30|23000| CA 90000|24|23000|| CA 99000| 40|24000|| NY| 83000| 36|19000|| NY| 79000| 53|15000| CA| 80000| 25|18000|| NY 91000| 50|21000| a)Create a data frame instance using the data given in the data file emp.csv and print its schema. b)Write a simple statement that will help you to get sum of salary for each department and show the outcome. c)How you will count the total number of states present in the dataset? d) Write the code snippet that will generate the following output? | department state|sum( salary)|sum(bonus)| |Finance NY | Marketing | NY | Sales | | Marketing ICA |Finance CA Sales NY (162000 91000 |81000 | 80000 |189000 |176000 | 34000 | 21000 | 23000 |18000 | 47000 | 30000 | | | I 1 e) Write the code snippet that generated the following outcome? || departmentsum_salary | avg_salary sum_bonus max_bonus | I | Sales |257000 |Finance | 351000 Marketing (171000 185666.66666666667 53000 187750.0 | 81000 85500.0 | 39000 123000 | 24000 | 21000