In this demo,
we'll cover a bunch of miscellaneous operations that
you can perform using Beautiful Soup.
We'll see how you can copy soup elements,
compare them for equality, use output formatters,
etc.
Set up the import statements for the modules and
libraries that you'll use for this demo.
Let's set up some very simple HTML code.
This markup contains two i_tags with the word Python.
I'm going to instantiate a soup object using this markup.
And here is what the prettified version looks like.
I'm now going to access the a_tag that exists within the soup object.
And once I have a reference to this a_tag,
I'll use the find_all function to find all i_tags within this a_tag.
There are two of them, which I assign to first_i.
It is Python.
And second_i also contains the word Python.
You can see that both of these i_tags are exactly the same.
Beautiful Soup allows you to use the double equality
comparison operator to compare tags.
First_i == second_i.
This returns True because the tag structure is exactly the same,
but they're actually different tags.
If you access the previous element for both of these tags,
you can see that they are different.
You can tell that first_i and second_i are different tags
because first_i is second_i will return False.
They don't occupy the same portion in Python memory,
but they can be compared for equality using the double equal to (==).
The double equal to equality comparison returns True
only if the tags are an exact match.
Here is some HTML code with two i_tags with the word Python within them,
but they have different classes applied.
The first i_tag has the class attribute value first,
and the second one has a class attribute second.
Let's access both of these i_tags into first_i and second_i.
Both of these have the same content, Python,
but first_i == second_i will return False.
The tags are not exactly the same.
The equality operator returns True only if the tags match perfectly.
If you want to make a copy of an existing Beautiful Soup tag,
you can use the copy function within the copy
module available built-in in Python.
A_copy is a copy of our a_tag that is not present within our parse tree.
You can use the equality operator, and you'll see that soup.a == a_copy.
But the copy is a different element occupying a
different portion in Python's memory.
A_copy is soup.a will return False.
The original a_tag, which is a part of our soup object, has a parent.
The name of the parent is the document itself.
The copy though has no parent.
It's not a part of our parse tree.
HTML markup can be converted to a soup object and back very easily.
Here is some HTML markup in the form of a string.
I can easily create a soup object using this HTML.
Simply instantiate Beautiful Soup, and that gives me a soup object.
If you want to convert the soup back to a string,
just parse the soup object into the built-in SDR function,
and that will give you the HTML string right back.
Beautiful Soup will automatically URL-encode your HTML.
Observe the ampersand present within this markup here.
Let's instantiate a soup object using the HTML parser.
And if you take a look at the contents of the b_tag,
the ampersand has been converted to a URL-encoded format.
Beautiful Soup automatically escapes ampersand and angular brackets.
Let's take a look at the a_tag, and the ampersand there has been escaped as well.
It's present as &.
Beautiful Soup performs this escaping so that it doesn't
inadvertently generate invalid HTML or XML.
You can also explicitly specify an output formatter for Beautiful Soup.
Here is a string in French.
Observe that the string contains a few HTML entities.
These are the HTML entities for e with an acute,
the less than and greater than signs.
I'm going to instantiate a soup object with the string.
And when I invoke the prettify function, I'll say formatter=None.
This will use no output formatting.
The string will be printed out as is, but might result in invalid HTML.
What's displayed here is invalid HTML, formatter=None.
By default though, soup.prettify will use the minimal formatter.
This will process the strings just enough to ensure that
Beautiful Soup generates valid HTML or XML.
If you specify formatter = 'html', Beautiful Soup will do some more processing.
It'll convert Unicode characters to HTML entities wherever possible.
If you specify formatter = 'html5' to soup.prettify,
it'll try and generate a valid HTML5 wherever possible.
HTML and HTML5 don't really make a difference for this particular markup.
Let's see something different, just the br_tag with no closing forward slash.
This is our soup object.
Let's format this using HTML.
And this is what HTML looks like.
We have the br with the forward slash.
Let's format this using HTML5,
and you'll find that the br exists without the forward slash.
That is valid HTML5.
You can have Beautiful Soup perform custom formatting on your strings.
Here is a custom function,
which converts a string to uppercase using the entity substitution module.
Here is our original markup that I've instantiated as a soup object.
It's initially in lowercase.
I invoke soup.prettify and specify my convert_uppercase as a formatter.
This custom formatter will format the string contents of my
HTML and convert them all to uppercase.