Choose the correct statement

In BP$,$ we update layers' parameters starting from the most inner layer to the most outer layer $($i$.$e$.,$ the output layer or prediction layer is the most outer layer$)$

Stochastic gradient descent and mini$-$batch stochastic gradient descent both require to compute the full gradients of the objective function

In deep learning applications, the training dataset is usually very large, so full gradients may not be computed efficiently and completely loaded into memory

Gradient descent can only be used for convex problems and it fails for nonconvex problems

$10$ point

Choose the correct statement

In $BP,$ we can tune the batch size, so that the computation of stochastic gradients can fit into the memory size.

The existing deep learning libraries, such as PyTorch, do not provide any functionality to automate the gradient computation and the model update

Learning rate $($or step size$)$ is not a hyper$-$parameter in training a deep learning model, and we do not need to tune its value to find a good performance.

In BP$,$ the computed stochastic gradients in each layer are all unbiased estimation of their full gradients