Sentence completion using N$-$gram:

Recommend the top $3$ words to complete the given sentence using N$-$gram language model. The goal is to demonstrate the relevance of recommended words based on the occurrence of Bigram within the corpus. Use all the instances in the dataset as a training corpus.

Test Sentence: Operating profit

Part II

Perform the below sequential tasks on the given dataset.

i$)$ Text Preprocessing:

Tokenization

Lowercasing

Stop Words Removal

Stemming

Lemmatization

ii$)$ Feature Extraction:

Use the pre$-$processed data from previous step and implement the below vectorization methods to extract features.

Word Embedding using TD$-$IDF

iii$)$ Similarity Analysis:

Use the vectorized representation from previous step and implement a method to identify and print the names of top two similar documents that exhibit significant similarity. Justify your choice of similarity metric and feature design. Visualize a subset of vector embedding in $2$D semantic space suitable for this use case. HINT: $($Use PCA for Dimensionality reduction$)$

Answer rating: 100% (QA)

The question seems to be a comprehensive one covering building an Ngram model and using it to predict the next word in a sentence as well as text preprocessing feature extraction and similarity analys... View the full answer