In a transformer's encoder, what role do positional encodings play, and how are they incorporated into the model?

Answer choices

Select only one option

Positional encodings are used to determine the order of tokens in the input sequence and are added to the output of the self$-$attention layer before feeding it into the feed$-$forward neural networks.

Positional encodings are applied after the feed$-$forward neural networks to determine the order of tokens in the output sequence, ensuring correct alignment with the input sequence.

Positional encodings are used to adjust the weights of self$-$attention mechanisms, emphasizing the importance of certain tokens based on their position in the sequence.

Positional encodings are introduced before the self$-$attention layers to provide the model with information about the order of tokens in the input sequence.