Create a program called Spider.java that implements the spider algorithm for exploring the Web. The spider algorithm essentially implements a breadth-first search on the Web graph, where nodes are html pages, and directed edges are hyperlinks from one web pages to web pages.

Question 2. Create a program called Spider.java that implements the spider algorithm for exploring the Web. The spider algorithm essentially implements a breadth-first search on the Welb graph, where nodes are html pages, and directed edges are hyperlinks from one web pages to web pages. Generally speaking the algorithm works as follows: Page D Page B Page A Page E Page C Page F INPUT: One or several seed URLs. . Other relevant information, e.g. a keyword to be searched an integer d indicating the depth of exploration (so you don't work with the whole Internet) OUTPUT certain data that are involved in all pages that are distance d away from the seed url(s) Data structures: (1) A list of unvisited URLs- seed this with one or more starting pages (2) A set of visited URLs-so you don't go around in circles (you should use a file for storing this list as it gets big) 3) Some rules for URLs you're not interested- so you don't index the whole Internet