Improving Accuracy

$\frac{0\mathrm{.}0}{5\mathrm{.}0}$ points $($graded$)$

We would like to try to improve the performance of the model by performing a mini grid search over hyper parameters $($note that a full grid search should include more values and combinations$).$ To this end, we will use our baseline model $($batch size $32,$ hidden size $10,$ learning rate $0\mathrm{.}1,$ momentum $0$ and the ReLU activation function$)$ and modify one parameter each time while keeping all others to the baseline. We will use the validation accuracy of the model after training for $10$ epochs. For the LeakyReLU activation function, use the default parameters from pyTorch $($negative$\_$slope $=0\mathrm{.}01).$

Note: If you run the model multiple times from the same script, make sure to initialize the numpy and pytorch random seeds to $12321$ before each run.

Which of the following modifications achieved the highest validation accuracy?

baseline $($no modifications$)$

batch size $64$

$$ learning rate $0\mathrm{.}01$