In this video,
we will look at the sections in Developer Tools that are of most interest
for use in identifying web media links, and in particular,
the Source and Element tabs.
While the Developer Tool section in the browser gives us a real wealth
of functionality, the ones we're mostly interested in from a web
scraping point of view for the moment are sources,
elements, and networks,
The Elements section allows us to view the browser
DOM in an interactive fashion.
We can use the pointer or the Inspect tool to appointed items on the page that
can help us identify where the items that we want to acquire are sitting in the
page and help us to identify the links. If we look at our earlier example of the
image and inspect it by clicking the pointer and then moving it over the image,
we can see the browser DOM updates and shows us exactly where in the document
the image was placed.
In this case,
the image is surrounded by a clickable href tag that brings us to
view more detail about the page, but we're interested, in this
case, in the image that is displayed,
and this is in the S or C or the source attribute. We can right‑click and
choose a number of options to help us know more about the media.
For example,
we can view the image itself in a separate tab to ensure it is, in fact,
what we require.
We can view in Sources panel to open the dedicated Sources panel and this
shows us information about the media object itself,
such as image size, file size, and content type. And finally,
we can get very useful information about where the object comes from
by copying the link address and we also then have the option to see
where the item is sitting in relation to the document itself by
using the Copy tool and selecting a number of different types of
selectors including CSS, JavaScript, and XPath.
While this can work well for items that are fixed in place like this image,
it's not so useful for media objects that are created in a dynamic
manner. Consider the very common use of a collection of images inside
a rotating carousel of other images. Here,
if we look at the DOM, we can see one image on show,
however,
depending on the button clicked a different image is
actually displayed to the user.
So when we're trying to identify the source of an image,
we need to know where each one is coming from and code or scrape accordingly.
In this example,
we can see that we have a collection of divs and each
one contains a different image.
Depending on the circular button selected,
each corresponding div has its display style set to be visible or to hide.
It may be the case that we want to select all of the images and that's fine,
but if we're only looking for the visible one,
then we need to look at the context of how the image is being displayed and
take that into consideration when planning our scrape.