Dropout. (a) Explain why dropout can be used to mitigate the overfitting problem. (b) Suppose we have

Question:

Dropout. (a) Explain why dropout can be used to mitigate the overfitting problem. (b) Suppose we have a simple two-layer neural network $y = W_{2} (W_{1} x + b_{1}) + b_{2}$ where $x \in R^{3}, W_{1} \in R^{2 \times 3}$ , $b_{1} \in R^{2}, W_{2} \in R^{2}$ and $b_{2} \in R$ . How many possible dropout networks are there that are not disconnected?