[Autogenerated] All right, let's get started with scrapie.
It's really important to make a clear distinction on
what scrapie is and what it isn't.
So what type of technology is scrapie?
Scrapie is a technology framework and not a library.
The definitions of the terms,
framework and library are often interchange mixed up
and even not correctly understood.
So to understand scrapie better,
let's explain the differences between what is a
framework and what is the library.
Let's first start off by looking at the characteristics
that make a technology of framework.
Essentially, a framework defines how your application works.
It gives your application the structure and boundaries it requires to operate.
And what about a library with the library?
Your application ends up using the library.
It invokes the library, and it is not defined by it.
When using a framework, your code is invoked by it.
When using a library, your code invokes the library,
so the approach is radically different.
Your application can only use one framework at a time.
You cannot design a nap that is shaped by two frameworks
that share the same scope or goals.
However, your application may use one or multiple libraries at the same time.
So this is a major difference between what is a framework and a library.
Now that we understand the differences between both concepts,
let's explore the two most important characteristics
that define scrapie as a framework.
Scrapie has a couple of very important features and
characteristics that it constitutes its fundamental pillars.
The first one is the capability that scrapie has to operate
spiders and overall execute a crawling strategy.
The synchronously.
This is perhaps the most important and defining feature of scrapie.
As without it,
it wouldn't be possible to run multiple spider jobs at the
same time with the high level of efficiency.
Scrapie is written with Twisted,
which is a well known network event driven framework for python.
To all, its logic is implemented in a non blocking concurrent code.
The other essential pillar of scrapie is its ability to have a
complete control over the data flow process.
This is possible by is modular design by having an engine at its core that is
able to process the crawl requests from the spiders and then these are
scheduled and passed to a downloader module to execute.
The results are returned as items and the process,
then repeats itself over and over.
Given that the engine executes all these requests synchronously,
there could be multiple control data flows being executed at the same time.
So given that scrapie is a synchronous and has full control of the data flow,
it presents some interesting advantages.
Essentially, scraping is fault tolerant,
which means that if a data flow or spider fails,
the system is able to recover due to the fact that the engine executes the jobs.
A synchronously scrapie can execute, task and run spiders at the same time,
which gives the overall impression of parallel execution.
Beyond that,
another great advantage of scrapie is that the framework is highly reliable.
This means that it gives you fine grain control of what you
can do with it and what you can achieve.
Finally,
it is definitely worth while highlighting this ______ is also built for speed,
which allows for fast logic execution.
Now that we understand what advantages gray app offers,
let's talk about its architecture.
Er