I have a basic understanding of fine$-$tuning models, but I need a detailed guide on the key steps for fine$-$tuning a pre$-$trained model in PyTorch. Specifically, I would like to know:

How to freeze and unfreeze layers for effective fine$-$tuning.

How to modify and add new layers for a specific task.

How to set different learning rates for different layers $($frozen vs$.$ fine$-$tuned$).$

Best practices to prevent overfitting during fine$-$tuning, especially with small datasets.

How to choose the right optimizer and learning rate scheduler for fine$-$tuning.

Any common challenges or mistakes to avoid when fine$-$tuning.

Could you provide a comprehensive guide, along with a PyTorch code example, demonstrating all these steps?