Let's talk about the first example mentioned in the last clip,
handling form submissions.
Like I mentioned,
a web form is something that allows a user to enter data that
is then sent to a server for processing.
If you're scraping a regular basic website,
this is not something that you have to worry about.
So what is something off the top of your head that
meets that particular criteria?
The search bar on an e‑commerce website like Walmart or
Amazon does seem to satisfy the criteria,
doesn't it?
You enter some term into the search bar,
it is sent back to the server for processing,
and you're shown a list of relevant products.
This isn't 100% true, but for our intents and purposes,
there are two types of forms, GET forms and POST forms.
A GET form is one where you can see the parameters in the URL.
To give you an example,
let's go to IMDb, or the Internet Movie Databases website
and search for a name, let's say Mark.
This is what comes up. Only a few names here.
We want more names and importantly exact name matches, so we'll
click here. We've arrived at the page we want. Now, let's check the
URL. The URL looks like this. There are multiple parameters here,
but let's focus on the queue parameter, which denotes the term
you're searching for.
As you can see, with the GET form,
the data you put into the form is kept in the URL. Let's
extract the search results from this page.
This must be pretty familiar to a lot of you, so let's get through this quickly.
First, we import the necessary packages, requests and BeautifulSoup.
Then, we use the get method to retrieve the data from our URL. We
then pass the content using the built‑in parser.
We go back to the web page, right‑click on the
relevant element, and click Inspect.
We then look at the information we get from inspecting the element and use
that in this code here to get the names of all the actors in the search
page, standard stuff. GET is nice and easy.
GET is our most pages work. And a useful thing about GET
forms is that you can share the URL to share the results.
But GET is not all there is.
In the next clip, we'll talk about something that will make it a
little more difficult to scrape data from a website.