$1.[$Points $45]$ Given the training examples. Answer the following questions.

Training Examples and their Labels$/$Tags:

Sequence $1$:

$$ The cat sits on the mat

$$ The $-->$ DET, cat $-->$ NOUN, sits$-->$ VERB, on$-->$ADP, the$-->$DET, mat$-->$NOUN.

Sequence $2$:

$$ A dog runs quickly

$$ A$-->$ DET, dog$-->$NOUN, runs$-->$VERB, quickly$-->$ADV

Sequence $3$:

$$ Birds fly in the sky

$$ Birds$-->$NOUN, fly$-->$VERB, in$-->$ADP, the$-->$DET, sky$-->$NOUN

$$

a$)[$Points $15]$ Compute Initial Label$/$Tag probabilities. Show detailed steps.

b$)[$Points $15]$ Compute Transition Probabilities. Show detailed steps.

c$)[$Points $15]$ Compute Emission Probabilities. Show detailed steps.

$$

$2.[$Points $35]$ Design a neural language model $($NNM$)$ with a fixed window $($e$.$g$.,$ w$=3)$ context using a feed$-$forward neural network. Draw a high$-$level diagram of your proposed model with parameter dimensions. Discuss details of each layer$$s input and output. How will you get the final output? How will you generate text using this language model?

$$

$3.[$Points $20]$ Address the following issues of your designed NNM in question $2,$ as below.

$$

a$.[$Points $10]$ Identify and discuss at least two limitations of your neural language model answered above. Provide a detailed rationale.

$$

b$.[$Points $10]$ Propose two improvements that could make the language model more accurate or robust. Provide a detailed rationale.