Question: Im using scrapy with python to crawl course information out of university website. I have the code but its not crawling information out and not

Im using scrapy with python to crawl course information out of university website. I have the code but its not crawling information out and not giving me any output. I would appreciate if someone can look at it and tell me what im doing wrong.

Im using scrapy with python to crawl course information out of universityspider.py

website. I have the code but its not crawling information out and

items.py

not giving me any output. I would appreciate if someone can lookoutput

Thank you

import scrapy from course.items import CourseItem cLass CourseSpider(scrapy.Spider): nameCourse allowed-domain = ['www . wayne . edu'] start-urls ['https://eiprod .wayne.edu/classschedule/201601/courses_new.cfm%3FSubj-PSL.htm1'] def parse(self, response) for scrape in response.xpath( //table/tr): item -CourseItem() item[ 'name'] scrape . xpath('' " "td [@id="homeContainer"]/div[2]/table/tbody/tr[1]/td [2]/a/text () " " " ) . extract () item['courseid'] scrape . xpath (.. " "tdl@id="honeContainer"]/div[2]/table/tbody/tr[1]/td [1]/text ( )""").extract() yield item import scrapy from course.items import CourseItem cLass CourseSpider(scrapy.Spider): nameCourse allowed-domain = ['www . wayne . edu'] start-urls ['https://eiprod .wayne.edu/classschedule/201601/courses_new.cfm%3FSubj-PSL.htm1'] def parse(self, response) for scrape in response.xpath( //table/tr): item -CourseItem() item[ 'name'] scrape . xpath('' " "td [@id="homeContainer"]/div[2]/table/tbody/tr[1]/td [2]/a/text () " " " ) . extract () item['courseid'] scrape . xpath (.. " "tdl@id="honeContainer"]/div[2]/table/tbody/tr[1]/td [1]/text ( )""").extract() yield item

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!