Let's look at the why and how we transition from Selenium to Beautiful Soup.
Firstly, the why of it.
Selenium itself is quite powerful in terms of locating elements and it
basically has everything you need for extracting data from HTML.
The problem is, it is slow.
Every single Selenium command goes through the JSON Wire HTTP
protocol and there is a substantial overhead.
In order to improve the performance of the HTML parsing part,
it is usually much faster to let Beautiful Soup parse the page source.
The process flow for the transition is as follows.
Navigate to the required website using Selenium.
Then perform the necessary browser actions,
such as filling in forms, search fields,
clicking on links, etc.
Once the desired data is on the page,
get the page source information from the driver.
And finally, pass it onto Beautiful Soup for parsing the HTML data.