In this video,
we will see how to use the requests library to handle larger files by
using the stream parameter. When we're dealing with simple media files
from a web page, downloading can be quite simple.
We issue a GET command, and the media downloads, job done.
There are times, however,
when we're dealing with larger files and files that need to be streamed,
and in this case we need to do a little bit more work. To help us along here,
we'll write a generic method that we pass in the file URL, and then we'll
use the stream parameter option for the requests library.
What this does is defer the downloading itself until we can access
the content part of the requests object. To start with, only the
response headers are downloaded, and the connection between the
client and server is kept open, thus giving us more control.
In this example,
we will use this to simply demonstrate how to stream save a large file.
However,
this can be extremely useful if you want to determine your next action
based on some information in the headers that you observe,
such as content size, type, last modified date,
etc.
Once we have the connection open, we iterate through the response
content, and as we receive chunks, we can write that to the file we
can then work with later as needed.