We've seen that you can specify different parsers to
parse HTML content in Beautiful Soup.
In this demo, we'll understand what the differences between these parsers are.
In order to see how different parsers work,
we'll work with some very simple markup,
which is not really valid HTML.
The markup starts with a b_tag.
It doesn't really contain the html_tag,
which forms the root of all HTML documents.
In addition, we have an i_tag here, which has no corresponding closing i_tag,
and there is a br_tag with no forward slash,
and there is a closing p_tag with no opening tag.
If you instantiate a Beautiful Soup object and parse
in this HTML code as your markup,
the default parser that will be used is the lxml parser,
provided lxml is installed on your machine.
If lxml is not installed, it'll use the built-in Python HTML parser.
We have lxml installed, and that's what will be used.
Lxml is an extremely fast and lenient parser.
It can parse broken HTML as well.
However, it has an external C dependency.
Lxml can also be used for XML documents.
Here is the HTML code when parsed using lxml.
Observe that it has added in the root html_tag, as well as a body_tag.
The original br_tag that did not have the forward slash
has had the forward slash inserted.
Our original i_tag had no corresponding closing i_tag.
The closing i_tag has been inserted as well.
There was a closing p_tag in our original markup
with no corresponding opening tag.
And this closing tag has been simply removed.
Lxml thought it was not useful.
And all of our markup has been enclosed within the html- and
body_tags to make it perfectly formed HTML.
If lxml is installed, that's the parser Beautiful Soup uses by default.
You can explicitly specify the lxml parser when you
instantiate your soup object as well.
The output HTML here is the same as what we saw earlier.
Let's now instantiate Beautiful Soup using a different parser.
This is the default built-in Python HTML parser.
This is a fairly fast parser.
It has decent speed, it's lenient, can work with broken HTML.
It's not as fast as lxml, and it's less lenient than html5lib.
If you parse in markup without the html- and the body_tags,
the HTML parser here does not add those in.
The open i_tag which did not have a corresponding
closing tag has been closed here, and the br_tag gets a forward slash.
If you want to generate valid HTML5, you should use the html5lib parser.
It's extremely lenient, and it parses pages just like a web browser does,
and it creates valid HTML5.
This is slow though and has an external Python dependency.
If your parsing speed doesn't really matter, this is a good parser to use.
Let's take a look at the HTML generated by this parser.
The html_tag has been added in, as has a header tag.
The head_tag is empty, but it's present.
A body_tag has also been added in, and our content is present within the body.
The br_tag gets a forward slash, and the p_tag has been closed.
It hasn't been removed.
And, finally,
you can always parse your HTML in the XML format by parsing in XML as a parser.
You need to have lxml installed though.