How can I implement L$1$ and L$2$ regularization in a PyTorch model? I am looking for a detailed explanation and code example that demonstrates how to apply both types of regularization during the training process. Specifically, I would like to understand:

How to set up L$1$ regularization manually in the loss function.

How to use L$2$ regularization $($weight decay$)$ with an optimizer.

A simple example using a linear regression model to illustrate these concepts.

Thank you for your help!