Question: Im using scrapy with python to crawl course information out of university website. I have the code but its not crawling information out and not
Im using scrapy with python to crawl course information out of university website. I have the code but its not crawling information out and not giving me any output. I would appreciate if someone can look at it and tell me what im doing wrong.
spider.py

items.py
output
Thank you
import scrapy from course.items import CourseItem cLass CourseSpider(scrapy.Spider): nameCourse allowed-domain = ['www . wayne . edu'] start-urls ['https://eiprod .wayne.edu/classschedule/201601/courses_new.cfm%3FSubj-PSL.htm1'] def parse(self, response) for scrape in response.xpath( //table/tr): item -CourseItem() item[ 'name'] scrape . xpath('' " "td [@id="homeContainer"]/div[2]/table/tbody/tr[1]/td [2]/a/text () " " " ) . extract () item['courseid'] scrape . xpath (.. " "tdl@id="honeContainer"]/div[2]/table/tbody/tr[1]/td [1]/text ( )""").extract() yield item import scrapy from course.items import CourseItem cLass CourseSpider(scrapy.Spider): nameCourse allowed-domain = ['www . wayne . edu'] start-urls ['https://eiprod .wayne.edu/classschedule/201601/courses_new.cfm%3FSubj-PSL.htm1'] def parse(self, response) for scrape in response.xpath( //table/tr): item -CourseItem() item[ 'name'] scrape . xpath('' " "td [@id="homeContainer"]/div[2]/table/tbody/tr[1]/td [2]/a/text () " " " ) . extract () item['courseid'] scrape . xpath (.. " "tdl@id="honeContainer"]/div[2]/table/tbody/tr[1]/td [1]/text ( )""").extract() yield item
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
