Question 1 and 2

Problem 1. In class we gave the following equation for the bigram probability of a sequence of words Wu}, ..., WU\"): k PT(W(1), __., WU\") = HPT.(W(i)lw(i1) = 10051)) (1) Using this formula, give an expression Ior the bigram probability of the sentence abab, where each character is treated as a word. Try to simplify the formula as much as possible. Problem 2. Let us suppose that there are two possible symbols/words in our language, a and b. There are three conditional distributions in the bigram model for this language, PT(W(")|W(FI) = a),Pr(W(i)|W('1) = b), and Pr(W()|W(i'1) = start), where start is the start symbol which begins any sentence. These conditional distributions are associated with the parameter vectors 9;, 632,, and 6mm, respectively (these parameter vectors were implicit in the previous problem). For the current problem, we will assume that these parameters are xed. Suppose that we are given a sentence W(1),...,W(k). We will use the notation away to denote the number of times that the symbol y occurs immediately following the symbol a: in the sentence. For example, 713\"; counts the number of times that symbol a occurs immediately following the symbol (1. Using Equation 1, give an expression for the probability of a sentence in our language: PT(W(1), u-3W(k)|'a,gba 'start) (2) The expression should make use of the nzny notation dened above. (Hint: the expres- sion should be analogous to the formula that we found for the likelihood of a corpus under a bag of words model.)