In each case, write a program implemented using Spark (either on AWS or Databricks), to:

Find the 5 most frequent and 5 least frequent (but present)t bi-grams for your dataset (only digits, not the decimal point A bi-gram is 2 successive digits/letters/etc.  For example, the string 938193 has 5 (93, 38, 81,19, 93).  The distribution would include:  93 – 2, and 81 - 1 . Assume that the data set is large enough so that bi-grams at the boundaries of nodes are not significant (most likely you will have only 1 mapper in any case since this is a very small data set, so it won’t be an issue.

Answer rating: 100% (QA)

Con... View the full answer