Question: Fill out the functions `n_word_types`, `n_word_tokens` in the [Preprocessing Block](#Preprocessing-Block). ***Note: you will have to rerun the `Preprocessing Block` cell every time you change its

Fill out the functions `n_word_types`, `n_word_tokens` in the [Preprocessing Block](#Preprocessing-Block). ***Note: you will have to rerun the `Preprocessing Block` cell every time you change its code for it to have any effect!***

print ("there are {} word types in the corpus".format(n_word_types(word_counts)))

print ("there are {} word tokens in the corpus".format(n_word_tokens(word_counts)))

questions is:

What is the difference between word types and tokens? Why are the number of tokens much higher than the number of types?

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!