Before we start,
let's look at the prerequisites and software requirements of the course.
Understandably,
you'll lead at least a basic to intermediate level of Python programming.
This is a course that deals with more advanced levels of web scraping,
so it is essential that you know how to handle the basic
steps. There are multiple courses on Pluralsight just a
search away that can help you with it, including mine.
That particular course, Scraping Dynamic Web Pages with Python and Selenium,
can be found in my profile.
The Python libraries we'll use primarily throughout the
course are Beautiful Soup and Requests.
A little bit of Selenium is also sprinkled onto the course. And with the
interactive environment, we'll be using the awesome Jupyter Notebook.
There's a good chance that you already have all of these packages. But in
case you don't, let's quickly go through how to install them. For Beautiful
Soup, all you need to do is type in pip install beautifulsoup4 into your
terminal or command line.
The same goes for the Requests library, pip install requests. For Selenium,
which is used to automate web browser interaction from Python, you'll first
install it like the others using the pip install selenium command, but
Selenium requires a driver to interface with the chosen browser. If you go
down on this page, you can see the download page links for four of the most
commonly used browsers. I use Chrome, so I'll click here, which takes me to
this page.
The web driver version you download is based on your Chrome version.
You can find that out by going to Help and then clicking on About
Google Chrome. You can see my version 80 and change,
so I'll be downloading this. Clicking on that takes you to a page where
you can find the zip version of the file. Download the version based on
your OS. Go ahead and unzip the file.
You should get an executable file or Chrome driver. To use this,
you can either make it accessible via the path variable of the
operating system. Or in case you get errors doing that,
you can directly specify the location of the Chrome driver. I'll make this
clearer once we get to the section of the course that requires Selenium. And as
for the Jupyter Notebook, you can either get it from the Anaconda distribution
or use the pip install jupyter command.
There you go.
We have everything we need.
Let's dig into the course.