Question: Can anyone help me with this problem? Python. I have to create a dictionary of bigrams which means it should be in form [previous word,
Can anyone help me with this problem? Python.
I have to create a dictionary of bigrams which means it should be in form [previous word, current word]
Building an MLE bigram model [Coding only: save code as problem2.py ]
Now, youll create an MLE bigram model, in much the same way as you created an MLE unigram model. I recommend writing the code again from scratch, however (except for the code initializing the mapping dictionary), so that you can test things as you go. The main differences between coding an MLE bigram model and a unigram model are: Select an appropriate data structure to store bigrams. Youll increment counts for a combination of word and previous word. This means youll need to keep track of what the previous word was. You will compute the probability of the current word based on the previous word count. Prob of curr word = count(prev word, curr word)/ count(previous word) Consider we observed the following word sequences:
finger remarked
finger on
finger on
finger in
finger .
Notice that "finger on " was observed twice. Also notice that the period is treated as a separate word. Given the information in this data structure, we can compute the probability p(on|finger) as 2/5 = 0.4. Similarly, we can compute the probability p(.|finger) as 1/5 = 0.2. When complete, add code to write 100 random (you can select a word and bigram term randomly) probabilities to bigram_probs.txt, one per line p(on|finger) = 0.4 p(.|finger) = 0.2
I also have the code for unigram model:
import sys filename = 'abc.txt' word_dict ={} total_count = 0 with open(filename, "r") as fp: for line in fp: words = line.split() for word in words: word = word.lower() if not word in word_dict: word_dict[word] = 1 else: word_dict[word] +=1 total_count +=1 probab = word_dict[word]/ total_count print(probab) fp.close() output_file = 'x.txt' with open(output_file, "w") as fs: fs.write(str(probab)) fs.close() Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
