Consider a neural network in which a vectored node v feeds into two distinct vectored nodes h1 and h2 computing different functions. The functions computed at the nodes are h1 = ReLU(W1v) and h2 = sigmoid(W2v). We do not know anything about the values of the variables in other parts of the network, but we know that h1 = [2,−1, 3]T and h2 = [0.2, 0.5, 0.3]T , that are connected to the node v = [2, 3, 5, 1]T . Furthermore, the loss gradients are ∂L

∂h1

= [−2, 1, 4]T and ∂L

∂h2

= [1, 3,−2]T , respectively. Show that the backpropagated loss gradient ∂L

∂v can be computed in terms of W1 and W2 as follows:

∂L

∂v

= WT 1

⎡

⎣

−2 0

4

⎤

⎦

+WT 2

⎡

⎣

0.16 0.75

−0.42

⎤

⎦

What are the sizes of W1, W2, and ∂L

∂v ?