Using the optimal learning rate with SGD$,$ we achieve a convergence rate of $1/$T

for strongly convex functions

for strongly convex functions, if we use a polynomial decay averaging

for strongly convex functions, if we use loss smoothing

for convex functions