$[3$ pts$]$ Explain why temporal data is not I.I.D$.($independent and identical distribution$)$ using example

of text data.

$[3$ pts$]$ Given the same input data, RNN model can generate different outputs at each time step.

Explain why this is possible using the output formula of RNN$.$

$[3$ pts$]$ Can we use bidirectional LSTM in the chapbot application? Explain the reason in detail.

$[5$ pts$]$ In p$.10$ of RNN slides, compute the error value $($using cross$-$entropy$)$ in the first time step

$($when input is $"h").$

$[5$ pts$]$ In LSTM$,$ the activation functions of gates $($forget$,$ input, and output$)$ are sigmoid functions.

Explain what will happen if we use ReLU instead.

$[5$ pts$]$ What are the problems with using vocabulary indexing in text processing. Explain it using the

example given in the slides.

In TF$-$IDF,

$[3$ pts$]$ when a $100-$word document contains the term "cat" $12$ times, compute TF value of 'cat'.

$[3pts$ The size of the corpus is $10$ million documents. If we assume there are $0\mathrm{.}3$ million documents

that contain the term "cat". Compute IDF $($use the log form$)$

$[2$ pt$]$ Compute TF$-$IDF of 'cat'

In CBOW,

$[3$ pts$]$ Can we use a vocabulary index method $($instead of a one$-$hot vector$)$ as an input to CBOW

$[2$ pts$]$ The activation function of the output layer should be softmax. Explain the reason.

$[3$ pts$]$ After training is done in CBOW, how do we extract the word embedding vector for a certain

word.

$[3$ pts$]$ In Skip$-$Gram, show that $F=$exp$(x)$ satisfies the following:

$F((w_i-w_j)tilde(w{)}_k)=\frac{F(w_i\text{t}\text{i}\text{l}\text{d}\text{e}(w{)}_k)}{F(w_j\text{t}\text{i}\text{l}\text{d}\text{e}(w{)}_k)}$