Question: what is the source code to write a scala program in spark shell on a macbook terminal to load a block_1.csv? will leave a thumbs
what is the source code to write a scala program in spark shell on a macbook terminal to load a block_1.csv? will leave a thumbs up for correct source code.

Write a Scala program in Spark Shell to load the block_1.csv dataset using spark.read.csv(), accessible from the Software Repository of the D2L course site, and perform the following: 1. Convert the dataset to RDD 2. Remove the heading (first record (line) in the dataset) 3. Convert the first two fields to integers 4. Convert other fields except the last one to doubles. Questions marks should be NaN. The last field should be converted to a Boolean. 5. Output an array of statistics for fields of type Double grouped by the last field with minimal passes
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
