[Autogenerated] in the previous demo,
we sell How to implement a spider using the scrapie thought Spider Class.
So now let's see how we can implement the same using crawlspace.
Er, let's have a look.
Let's create across Spider by running the Gen Spider Command as follows.
Executing this command creates the quote crawler dot Python file.
This file includes this boilerplate code.
Let's also add the required libraries and also at a reference to quote item.
Let's modify allowed domains and assign it the value of the end.
And let's set start URL SSHD to the first page.
Now notice the rules.
This will be used by cross speeder to follow all the links found on
the site according to how each rule is defined.
Let's change the value of items to page,
as that indicates the partial path of the URL that the cross spider will follow.
For each page crawled.
The parts page method will be invoked,
so let's change the name of parse item to parse page.
Let's also remove this code and evoke extract data.
Extract data looks exactly like it did in our previous demo.
An instance of quotes.
Item is created and then for each quote found within the mark of the quote.
Author and tags are extracted just like we did in the previous demo.
Then right y-axis is called and the info is written to this.
Now let's run this creepy crawl command for this spider to see what happens.
As you can see,
it is crawling pages that the spider in the previous demo did not crawl.
Let's check the results.
As you can see, it's a lot more entries than what our previous spider retreat.
So why is that?
Let's browse through the tags to see notice how there are
pages here that we did not parse before,
which contains essentially the same content but organized by tags.
So let's change this regular expression from page to tag to see if
it makes any difference in the number of results.
If we execute the spider and check the results,
we can see that the spider has scraped all the pages and tags from the site,
but organized slightly different.
Now let's tell the spider to crawl the cycling's,
but instead by specifying the CSS selector,
we want the spider to follow here.
We can get the tax collector by inspecting it from the markup,
and we can add it to the crawling rule by using the restrict
CSS option and indicating the selector.
Now we can run the spider again and check the results.
The results are the same.
The spider has crawled all the links found on the tag elements,
but in this case using a CSS electoral rules.
If we want to achieve the same functionality as with our
previous spider and have the spider, just follow the next page and not the tags.
Always simply need to do is to change the restrict CSS selector to ally dot.
Next, This will give us a reduce set of results,
just like the previous spider did.
So as you have noticed, crawl Spider has a much broader scope,
then scrape it out.
Spider cross-platform follows by default all the links found on a site,
whereas with scrapie that spider,
you have to specifically indicate what pages it should crawl.
This is the main difference between both classes and how they work