We could represent the word with one$-$hot encoding in a language model instead of ascii representations.

That is$,$ we represent the word w with:

$\backslash |$ V $\backslash |$

z $\}|\{$

$[0,0,...,1,...,0,0]$

where V is our vocabulary and $\backslash |$ V $\backslash |$ is the size of it$.$