Question: In python, Need help writing loop for a TF-IDF Vectorizer that goes through the iterations of max_features, min_df, max_df and chooses the best parameters for

In python,

Need help writing loop for a TF-IDF Vectorizer that goes through the iterations of max_features, min_df, max_df and chooses the best parameters for logistic regression. The parameters should give the max accuracy, precision, recall, F-1. When sending the data into test and train sets, the random seed is 100, 50% split size. Loop should go through 10 iterations. I have provided the most parts of the loop, and I need help with the syntax and process of the loop.

vectorizer = CountVectorizer(max_features=1601,min_df=3, max_df=1.0, stop_words=stopwords.words('english'))

X = vectorizer.fit_transform(norm_corpus).toarray()

transformer = TfidfTransformer()

X=transformer.fit_transform(X).toarray()

text_train, text_test,sent_train, sent_test = train_test_split (X,y,test_size=0.5, random_state=100)

classifier = LogisticRegression()

classifier.fit(text_train,sent_train)

sent_pred = classifier.predict(text_test)

cm = confusion_matrix(sent_test,sent_pred)

tot=cm[0][0]+cm[1][1]

print (tot)

#Model accuracy

print(tot/cm.sum())

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Programming Questions!