Evaluating the SGD optimization and training$/$evaluating the DAN model

SGD$\_$optimization $(25/25)$

$==$

All calculated gradients are correct

$=$

SGD converged to the optimum

DAN$\_$dev $(50/50)$

$==$

Accuracy: $691/872=0\mathrm{.}792431$;

Precision $($fraction of predicted positives that are correct$)$: $348/433=0\mathrm{.}803695$;

Recall $($fraction of true positives predicted correctly$)$: $348/444=0\mathrm{.}783784$;

F$1($harmonic mean of precision and recall$)$: $0\mathrm{.}793615$;

Time for training and evaluation: $89\mathrm{.}37$ seconds

DAN$\_$dev$\_$typo $(15/25)$

$==$

Accuracy: $627/872=0\mathrm{.}719037$;

Precision $($fraction of predicted positives that are correct$)$: $328/457=0\mathrm{.}717724$;

Recall $($fraction of true positives predicted correctly$)$: $328/444=0\mathrm{.}738739$;

F$1($harmonic mean of precision and recall$)$: $0\mathrm{.}728080$;

Time for training and evaluation: $187\mathrm{.}56$ seconds $-$ for the typo accuracy i$\text{'}$m getting $70\%$ but i want to increase this to $77\%$ here is my code $-$ class PrefixEmbeddings:

def $\_\_$init$\_\_($self$,$ word$\_$indexer, embeddings, embedding$\_$dim, prefix$\_$length $=3)$:

self.word$\_$indexer $=$ word$\_$indexer

self.embedding$\_$dim $=$ embedding$\_$dim

self.prefix$\_$length $=$ prefix$\_$length

self.prefix$\_$indexer $=\{\}$

self.embeddings $=$ torch.tensor$($embeddings$,$ dtype $=$ torch.float$32)$

for word in self.word$\_$indexer.objs$\_$to$\_$ints.keys$()$: # Iterate over words

idx $=$ self.word$\_$indexer.index$\_$of$($word$)$

if idx $!=-1$: # Check if the word exists

prefix $=$ word$[$:prefix$\_$length$]$

if prefix not in self.prefix$\_$indexer:

self.prefix$\_$indexer$[$prefix$]=[]$

self.prefix$\_$indexer$[$prefix$].$append$($self$.$embeddings$[$idx$])$ # Append the embedding

self.prefix$\_$indexer$[$prefix$].$append$($embeddings$[$idx$])$

self.prefix$\_$embeddings $=\{\}$

for prefix, word$\_$embeddings in self.prefix$\_$indexer.items$()$:

avg$\_$embedding $=$ torch.mean$($torch$.$stack$($word$\_$embeddings$),$ dim $=0)$

self.prefix$\_$embeddings$[$prefix$]=$ avg$\_$embedding

def get$\_$embedding$($self$,$ word$)$:

for prefix$\_$len in range$($self$.$prefix$\_$length, $1,-1)$:

prefix $=$ word$[$:prefix$\_$len$]$

if prefix in self.prefix$\_$embeddings:

return self.prefix$\_$embeddings$[$prefix$]$

return torch.randn$($self$.$embedding$\_$dim$)$