[Autogenerated] Now let's talk about the crawl spider and go
over its most important properties and methods.
The rules property is a list of one or more rule objects.
Each rule object defines a certain behavior for crawling the site.
The par start URL method is called for start, URL responses.
The link extractor object defines how links will be
extracted from each crawled page.
Now let's have a look at an implementation example of this.
Here is a cross spider class example with some rules.
We first import the Scrapy library and also specifically
crawl spider rule and link extractor.
Then just like we did on the previous gravy dot spider example,
we specify the name allowed domains and start URL properties.
Next we define the rules that the spider will be using for crawling.
It contains one rule that specifies the link extractor on the product dot PHP
URL which triggers the execution of the parts item method.
The parse item method creates a new Scrapy dot item object which
contains the name and link tax properties the name property of the
Scrapy dot item is assigned the value of each product name and the link
text property is assigned each meta value of the link text ad review
within the markup the parse item,
then returns the value of the Scrapy dot item object