Consider the fully recurrent network architecture $($without output activation and bias units$)$ defined as

$s(t)=Wx(t)+Ra(t-1)$

$a(t)=f(s(t))$

hat$(y)(t)=Va(t)$

with input vectors $x(t),$ hidden pre$-$activation vectors $s(t),$ hidden activation vectors $a(t),$ activation function $f(*)$ and parameter matrices $R,W,V.$ Let $L(t)=L(y(t),$hat$(y)(t))$ denote the loss function at time $t$ and let $L={\displaystyle \underset{t=1}{\overset{T}{}}}L(t)$ denote the total loss. We use denominator$-$layout convention, i$.$e$.,(t)=\frac{\text{d}\text{e}\text{l}\text{L}}{\text{d}\text{e}\text{l}\text{s}(t)}$ is a column vector. Which of the following statements are true?

a$.$ The asymptotic complexity of BPTT is $O(T^2).$

b$.$ The gradient of the loss with respect,to the input weights $W$ can be written as $\frac{\text{d}\text{e}\text{l}\text{L}}{\text{d}\text{e}\text{l}\text{W}}={\displaystyle \underset{t=1}{\overset{T}{}}}(t)x^{TT}(t).$

c$.$ BPTT is a common regularization technique for recurrent neural networks.

d$.$ The gradient of the loss with respect to the recurrent weights $R$ can be written as $\frac{\text{d}\text{e}\text{l}\text{L}}{\text{d}\text{e}\text{l}\text{R}}={\displaystyle \underset{t=1}{\overset{T}{}}}(t)a^{TT}(t-1)$

e$.$ The deltas fulfill the recursive relation $(t)=$diag$(f^{\text{'}}(s(t)))(V^{TT}\frac{\text{d}\text{e}\text{l}\text{L}(t)}{\text{d}\text{e}\text{l}(hat(y))(t)}+R^{TT}(t-1)).$