Hint:

We say two quantities are correlated when one quantity increases $($decreases$)$ and the other increases $($decreases$).$

We say two quantities are anticorrelated when one quantity increases $($decreases$)$ and the other decreases $($increases$).$

We say two quantities are uncorrelated when there is no obvious relation between them.

In $[15]$: # THIS IS JUST A TEMPLATE. COPY THE VARIABLES AND CHANGE THE VALUES. DO NOT CHANGE VARIABLE NAMES!

answer$2\_1=\text{'}$A$\text{'}$

answer$2{?}^{-}{?}^2{?}^{\text{'}}{B}^{\text{'}}$

answer$2{?}^{-}3={?}^{\text{'}}{C}^{\text{'}}$

answer$2\_4={?}^{-}{D}^{\text{'}}$

###

### YOUR CODE HERE

###

In $[16]$: answers $\text{'},\text{'}$B$\text{'},\text{'}$C$\text{'},\text{'}$D$\text{'},\text{'}$E$\text{'}]$

assert answer$2\_1$ in answers, 'answer$2\_1$ should be one of $\{\}\text{'}.$format$($answers$)$

answers $\text{'},\text{'}$B$\text{'},\text{'}$C$\text{'},\text{'}$D$\text{'}]$

assert answer$2\_2$ in answers, 'answer$2\_2$ should be one of $\{\}\text{'}.$format$($answers$)$

answers $\text{'},\text{'}$B$\text{'},\text{'}$C$\text{'}]$

assert answer$2\_3$ in answers, 'answer$2\_3$ should be one of $\{\}\text{'}.$format$($answers$)$

answers $\text{'},\text{'}$B$\text{'},\text{'}$C$\text{'}]$

assert answer$2\_4$ in answers, 'answer$2\_4$ should be one of $\{\}\text{'}.$format$($answers$)$Shuffling

$1$ point possible $($graded$)$

Which of the following are true about the shuffle operation in Spark?

You are encouraged to read more about shuffle here.

All Spark operations involve shuffling

Shuffling requires large amounts of network communication which makes the operation expensive

Shuffling is Spark's mechanism for re$-$distributing data across partitions

To avoid shuffling, we should think about ways to leverage existing partitions. For example, leveraging

partial aggregation to reduce data transfer

No communication between worker nodes is required for the shuffling operation