And this demo where we got started using Beautiful Soup to parse web
pages brings us to the very end of this module.
We started this module off by understanding what web
scraping is and how it can be useful.
We saw that web scraping involves two phases.
The first step is to, first, fetch the web content via HTTP requests.
We saw how we could make HTTP requests to fetch content using
three different Python client libraries,
the high-level requests API, httplib2, and urllib.
Once the HTML content was available to us programmatically,
we saw that we could parse it using regular expressions.
Regular expressions are a pattern matching technique to allow us to
search for specific strings that match the pattern.
We saw how we could parse HTML using regular expressions,
but we also saw that regular expressions are kind of hard to work with.
Regular expressions don't really understand the HTML
parse tree structure and markup.
Beautiful Soup does,
which is why Beautiful Soup is a great way to extract content from HTML.
In the next module,
we'll see how we can use Beautiful Soup objects to navigate the HTML parse tree.