Question: Remove Stop Words 1 point possible ( graded ) Try to implement stop words removal in your feature engineering code. Specifically, load the file stopwords.txt

Remove Stop Words
1 point possible (graded)
Try to implement stop words removal in your feature engineering code. Specifically, load the file stopwords.txt, remove the words in the file from your dictionary by editing the bag_of_words function , and use features constructed from the new dictionary to train your model and make predictions.
Compare your result in the testing data on Pegasos algorithm using T =25 and L =0.01 when you remove the words in stopwords.txt from your dictionary.
Hint: Instead of replacing the feature matrix with zero columns on stop words, you can modify the bag_of_words function to prevent adding stopwords to the dictionary
Accuracy on the test set using the original dictionary: 0.8020
Accuracy on the test set using the dictionary with stop words removed:

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!