In this video,
we will talk about how to deal with large media files that
we might want to acquire from the web.
One of the characteristics of web media that you'll come across is
that a lot of it can be quite large in size.
The implications of this are threefold.
First, it can consume a lot of bandwidth to download,
2, it can take a lot of storage, and number 3,
it can consume a lot of resources to process the data once you have it.
When considering how to deal with these different kinds of files,
we need to take into account both the cost of the
data in motion and at rest and, of course,
the process of acquiring such a large amount of data in the first place.
While the internet and its supporting technologies
have come a long way over the years,
one of the key lessons we've learned is to assume that
things will go wrong and to plan for them.
Working with large web media files is no different.
When acquiring large files, we have to assume,
for example, that we may, for one reason or another,
lose our connection to the server while sending the file to our client and
that we may end up with a partially incomplete file.
The best way of ensuring we can handle these kinds of situations is to
assume that the transfer is going to fail and ensure that we have the
ability to pause our data transfer and resume it from the point that we
left off at some later stage.
For web files, luckily,
there is a mechanism ready for us to use that helps us to download a file
from any point in the data stream that makes up the file.
This is known as these accept‑ranges header and is supported by all modern
browsers and the majority of internet servers today.
The concept of auto‑resume is very straightforward.
We know that when we download a file,
we are downloading it is a series of bytes in a stream.
For systems that support accept‑range, we can not only tell the web
server that we want a copy of a particular file,
but we can also tell it exactly the position in the file stream that we
want it to start sending us data from. Another useful tool we can use for
working with large files is the HEAD command. This allows us to query the
web server for details of a large file,
such as its size before we start downloading it. Using the HEAD and the file
size together with the accept‑range header allows us to recover confidently
from failed downloads by resuming our download exactly from the point that we
encountered the problem in the first place. We will look at an example of
handling this issue in a later module.