We will define a class to model the fully connected multi layer network. The network parameters will be a dictionary accesssed with keys $"$W$1","$b$1","$W$2","$b$2",$ etc.

ensure very small values close to zero for faster training. The requires$\_$grad will be needed for tracking gradient. We limit the precision to torch.float $32($optional$).$

Bias values are initialized with $0.$ For example a bias vector of dimensions $31$ is initialized as $,$ requires$\_$grad$=$True, dtype$=$torch.float$32)$

class MultiLayerNetwork$()$:

def$\_$init$\_($self$,$ net$\_$dims, activation$\_$list$=$None$)$:

# Activation functions can be provided through activation$\_$list

# Default activations are relu for all layers

# except for the last layer which is Softmax

self.allowed$\_$activations $=[\text{'}$relu$\text{'},$ 'sigmoid', 'tanh'$]$

self.final$\_$activation $=$ 'softmax'

self.num$\_$layers $=$ len$($net$\_$dims$)-1$

if activation$\_$list is None:

self.activation$\_$list self.num$\_$layers$-1)$ 'softmax'