Question: 1. Write a python program to a. Extract the contents (excluding any tags) from two websites (https://en.wikipedia.org/wiki/Web_mining&https://en.wikipedia.org/wiki/Data_ mining). b. Remove stopwords [using Spacy Module] (including

1. Write a python program to a. Extract the contents (excluding any tags) from two websites (https://en.wikipedia.org/wiki/Web_mining&https://en.wikipedia.org/wiki/Data_ mining). b. Remove stopwords [using Spacy Module] (including the special characters/symbols) from the contents retrieved from those two URLs and save the contents in two separate .txt file.  [List of additional Stop words to be considered = [dot, comma, singlequote, double quote, question mark, brackets [square, parentheses, curly, angle], exclamation mark]] c. Display the POS tag (sentence-wise) for all the stopwords (excluding the special character/symbols), which are removed from the content, using pandas dataframe as per the format given below: Original Sentence List of Stopwords POSTags Web mining is the application of data mining techniques to discover patterns from the World Wide Web. is the of to from the VBZ DT IN TO IN DT Assessment - 1 CSE 3024: Web Mining Page 2 d. Display the Term-Document incidence matrix using Boolean, Bag-of-words and Complete representation (Use pandas dataframe). Prepare three separate table, one for each type of representation as per the format given below: Terms DOC1 DOC2 Web 5 0 Data 0 1 e. Input a search a query (preferably a sentence) and compare the contents of the both pages with the processed query. Display the similarity result based on highest frequency matching count of the term. 2. Write a python program to prepare the Word Clouds representation based on the content present in the two document files prepared in Q.No. 1. A sample Word Clouds representation is provided below for reference. 3. Write a python program to show the implementation of sentence paraphrasing through synonyms (retaining semantic meaning) for the following four sentences. Display at least three other paraphrased sentences for each sentence mentioned below. a. The quick brown fox jumps over the lazy dog b. We can rewrite history as much as we like. c. Once you know all the elements, it’s not difficult to pull together a sentence. d. The incessant ticking and chiming echoed off the weathered walls of the clock repair shop.

Step by Step Solution

★★★★★

3.53 Rating (160 Votes )

There are 3 Steps involved in it

1 Expert Approved Answer

Step: 1 Unlock

import required modules import requests get URL page requestsgethttpsenwikipediaorgwik... View full answer

Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock

Step: 3 Unlock

Students Have Also Explored These Related Operating System Questions!

Find the minimum variance portfolio with expected rate of return of 8% using 5 assets with expected rates of return = [0.10,0.12, 0.05, 0.03, 0.15] , standard deviation = = [0.4, 0.45, 0.15,0.1,...

In this assignment, you will write a Python program that asks the user for a list of words, and then prints the shortest word, the longest word, and the average length of the words. Here is a sample...

Python 3 Write a python program that generates 1,000 random integers between 0 and 9 and displays the count for each number (Hint: Use a list of ten integers, say counts, to store the counts for the...

Using alphabetical order, construct a binary search tree for the words in the sentence "The quick brown fox jumps over the lazy dog."

This project implements an interface and follows Array collections. With Array collections there are methods used such as toString() or something like isFull(). Helper methods can be used for the...

import java.util.Scanner; // Use a scanner and read each word from a string containing multiple words. Print each word in reverse order public class ReverseWord { public static void main(String[]...

The purchase schedule for Laundryman's Corporation is as follow: The inventory balance as of the beginning of the year was $35,000 (500 units @ $70 each). During the year ended December 31, the...

Software to detect fraud in consumer phone cards tracks the number of metropolitan areas where calls originate each day. It is found that 1% of the legitimate users originate calls from two or more...

After studying Iris Hamson's credit analysis, George Davies is considering whether he can increase the holding period return on Yucatan Resort's excess cash holdings (which are held in pesos) by...

The following transactions for Nielsen Architects occurred during December of the current year. Follow the step-by-step instructions to record the transactions, process the adjusting entries, and...

The risk free rate is 2.4% and the market risk premium is 5.0%. A stock has a beta of 1.2, what is its expected return of the stock? (Enter your answers as a percentage)

Among the strategies that Amazon has implemented is the cost reduction strategy, focus and leadership in reducing costsIn addition to the strategy of maintaining customer loyalty and using technology...

Your organization wants to encourage more collaboration and communication in the organization. The product team wants to collaborate with marketing and the sales team beyond the usual email system....

Din frsta uppgift r att lsa, reflektera och analysera caset som finns bifogat, lngst ner i den hr texten, fr INL1. Caset beskriver hur Vanguard Electronics frndrade sin struktur genom att utveckla...

General Grapplegrommets is an aerospace parts manufacturer founded in 1972 in Wembley, Saskatchewan, that supplies landing gear components for a range of small passenger aircraft. Their head office...

Part 2: Average Annual Salary of Ohio Employees. 6. Suppose the average annual salary in dollars, A, of an Ohio employee is a function of the time of years since 2010, t. An approximate formula is A...

Based on the following projected returns compare investment performance under the following scenarios. The figures are projected quarterly returns, are over an 8 year investment horizon and involve...

Describe the Operations (+,,*,/) that can cause negligible addition (NA), error magnification (EM), or subtractive cancellation (SC) in calculating ?((x^2)+1) - x . Give the range of where they might...

Perform experiments to evaluate the efficiency of the remove method of Pythons list class, as we did for insert on page 205. Use known values so that all removals occur either at the beginning,...

Implement a function that reverses a list of elements by pushing them onto a stack in one order, and writing them back to the list in reversed order.

Draw a compressed trie for the strings given in the previous problem. Previous problem Draw a standard trie for the following set of strings: {abab, baba, ccccc, bbaaaa, caa, bbaacc, cbcc, cbca}.

The data shown in the following table were obtained from the 1990 Census. Included is information on 26 randomly selected Metropolitan Statistical Areas (MSAs). Of interest are factors that...

Researchers have studied the ecology of ponds in rural Bangladesh. Of particular interest in such studies are the zooplankton, phytoplankton, and copepod counts (per liter of water) in these ponds....

For married couples with one or more offspring, a demographic study was conducted to determine the effect of the husband's annual income (at marriage) on the time (in months) between marriage and the...