Question: You will learn to implement a POS tagger using CRF . Complete each of the following tasks. Load NLTK Treebank tagged sentences using nltk .

You will learn to implement a POS tagger using CRF. Complete each of the following tasks.
Load NLTK Treebank tagged sentences using nltk.corpus.treebank.tagged_sents().
Use first 80% of the sentences for training and the remaining 20% for the testing.
Extract the word and the tag from each of the sentences and create a vocabulary of all
the words and a set of all tags.
Build the following feature set for each token/word:
The current token/word
Is the word a number? (boolean value)
Does the word contain any hyphens? (boolean value)
Is the word all uppercase? (boolean value)
Does the word have any uppercase letters? (boolean value)
Is the word all lowercase? (boolean value)
Length of the word
Bigrams of the word
Use the CRF model from sklearn_crfsuite library ?6, and train it with feature set built
above.
Evaluate the performance of the model in terms of accuracy on the test set.
 You will learn to implement a POS tagger using CRF. Complete

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!