[Autogenerated] hi and welcome back in this module,
we're going to be looking at how to scaffold and run our first gray app.
WebCrawler project.
Previously,
we saw how tricky things could get when we need to fetch
details from a website from more than one level.
This is where scrapie comes into the rescue.
So what are we going to cover in this module?
First, To set the context right,
we'll have an introduction to scrapie so we'll know
what it is and what advantages it has.
Then we'll explore the architecture of scrapie,
which includes very nice features,
such as the ability to limit download requests,
to manage concurrent connections and also to handle automatic throttling.
Beyond that, scrapie is also made up of spiders items,
a schedule, er, a downloader.
Besides the main engine.
Following that, we'll explore in detail beautiful soup,
what are its main objects and what advantages it has
compared to other parsing solutions.
Then we will create a new scrapie project from scratch.
Sounds really awesome.
So let's dive in