Question: In this assignment, you will try to gather text data from open data source via web scraping or API. After that you need to clean

In this assignment, you will try to gather text data from open data source via web scraping or API. After that you need to clean the text data and syntactic analysis of the data. Question 1 Write a python program to collect text data from either of the following sources and save the data into a csv file: (1) Collect all the customer reviews of the product 2019 Dell labtop on amazon. (2) Collect the top 100 User Reviews of the film Joker from IMDB. (3) Collect the abstracts of the top 100 research papers by using the query natural language processing from CiteSeerX. (4) Collect the top 100 tweets by using hashtag "#CovidVaccine" from Twitter. Write the code using python

Write a python program to clean the text data you collected above and save the data in a new column in the csv file. The data cleaning steps include:

(1) Remove noise, such as special characters and punctuations.

(2) Remove numbers.

(3) Remove stopwords by using the stopwords list.

(4) Lowercase all texts

(5) Stemming.

(6) Lemmatization.

Write a python program to conduct syntax and structure analysis of the clean text you just saved above. The syntax and structure analysis includes:

(1) Parts of Speech (POS) Tagging: Tag Parts of Speech of each word in the text, and calculate the total number of N(oun), V(erb), Adj(ective), Adv(erb), respectively.

(2) Constituency Parsing and Dependency Parsing: print out the constituency parsing trees and dependency parsing trees of all the sentences. Using one sentence as an example to explain your understanding about the constituency parsing tree and dependency parsing tree.

(3) Named Entity Recognition: Extract all the entities such as person names, organizations, locations, product names, and date from the clean texts, calculate the count of each entity.

Note: Questions 1&2 were answered yesterday, but I was told that third question needs update. I guess it was assigned to another expert. I have posted questions 1&2 in order to know how to go about question 3. I will be glad if I can get it in the next one hour. Thanks.

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!