In the earlier module,
we discussed in a lot of detail how we can fetch HTML content from a web site.
We'll now talk about parsing HTML content.
Web scraping involves fetching content and parsing content.
We've discussed fetching content.
Let's turn our attention to parsing content using HTML parsing and DOM parsing.
DOM parsing usually refers to using JavaScript within
a web browser to navigate the DOM, that is the Document Object Model,
that is a tree representation of your HTML.
The DOM is dynamic and can be modified using
JavaScript running within your browser.
Beautiful Soup performs HTML parsing,
and that's what we'll focus our attention on.
If you want to parse and extract content from HTML in Python,
Beautiful Soup is essentially the only game in town.
There are other alternatives that you could use to parse web content.
You can use regular expressions, you can use grep utilities,
but they're not really tailored for HTML.
If you're working with HTML, Beautiful Soup is what you'll want to use,
a Python package for parsing HTML as well as XML hierarchical structures.
The cool thing about Beautiful Soup is that its parser
libraries clean up malformed HTML.
It's quite common in the real world to have HTML
pages which don't have closing tags, which are really very,
very messy.
Beautiful Soup will clean up the HTML for you and then parse it.
Beautiful Soup is an easy-to-use and intuitive Python package,
and it mitigates the weaknesses of regular expressions for parsing.
When you parse your HTML using Beautiful Soup,
it forms a parse tree of your entire HTML contents by default.
You can navigate up and down this parse tree,
go sideways, and also perform search and filtering operations.
Beautiful Soup is relatively simple and intuitive to use,
and it's also robust to problems in markup being parsed.
Markup is very unstructured and is unclean normally out in the real world.
Beautiful Soup takes care of that for you.