n class, we showed that a com$-$ puting a regular self$-$attention layer takes O$($T$2)$ running time for an input with T tokens. One alternative is to use $$linear self$-$attention$.$ In the simplest form, this is identical to the standard dot$-$product self$-$attention discussed in the class and lecture notes, except that the exponentials in the rowwise$-$softmax operation softmax$($QK$)$ are dropped;