Question: You will learn to implement a POS tagger using CRF . Complete each of the following tasks. Load NLTK Treebank tagged sentences using nltk .
You will learn to implement a POS tagger using CRF Complete each of the following tasks.
Load NLTK Treebank tagged sentences using nltkcorpus.treebank.taggedsents
Use first of the sentences for training and the remaining for the testing.
Extract the word and the tag from each of the sentences and create a vocabulary of all
the words and a set of all tags.
Build the following feature set for each tokenword:
The current tokenword
Is the word a number? boolean value
Does the word contain any hyphens? boolean value
Is the word all uppercase? boolean value
Does the word have any uppercase letters? boolean value
Is the word all lowercase? boolean value
Length of the word
Bigrams of the word
Use the CRF model from sklearncrfsuite library and train it with feature set built
above.
Evaluate the performance of the model in terms of accuracy on the test set.
Step by Step Solution
There are 3 Steps involved in it
1 Expert Approved Answer
Step: 1 Unlock
Question Has Been Solved by an Expert!
Get step-by-step solutions from verified subject matter experts
Step: 2 Unlock
Step: 3 Unlock
