Question: Fill out the functions `n_word_types`, `n_word_tokens` in the [Preprocessing Block](#Preprocessing-Block). ***Note: you will have to rerun the `Preprocessing Block` cell every time you change its
Fill out the functions `n_word_types`, `n_word_tokens` in the [Preprocessing Block](#Preprocessing-Block). ***Note: you will have to rerun the `Preprocessing Block` cell every time you change its code for it to have any effect!***
print ("there are {} word types in the corpus".format(n_word_types(word_counts)))
print ("there are {} word tokens in the corpus".format(n_word_tokens(word_counts)))
questions is:
What is the difference between word types and tokens? Why are the number of tokens much higher than the number of types?
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
