In an earlier module,
I used hiQ Labs as an example of a web scraping business model,
and mentioned that hiQ had been part of an important recent legal battle.
In this section, we'll review the details of the case.
Again, I'm a big data engineer, not an attorney.
The legal issues quickly get complex, but I'll review the case for you.
Remember,
hiQ Lab's business model involves scraping publicly available LinkedIn data
to create talent analytics that can determine when employees might leave for
another company or to build a skills inventory.
The court had to evaluate several facts and questions before ruling on the case.
It started when LinkedIn sent hiQ a cease and desist letter,
demanding that hiQ stop accessing and copying data from LinkedIn's server.
LinkedIn claimed the Computer Fraud and Abuse Act,
or CFAA, prohibited unauthorized access.
That's a 1986 American law meant to criminalize hacking,
breaking into private computer systems to access nonpublic information.
HiQ turned the tables and filed suit against LinkedIn,
claiming the survival of its business was under threat.
HiQ was concerned that LinkedIn would use technical means to block its access,
harming its business, and so they asked the court to stop LinkedIn.
In the fall of 2019, a US appeals court in California ruled in favor of hiQ and
against LinkedIn. The court ruled that hiQ's automated access of publicly
available data was not in violation of the CFAA act.
Now this ruling only applies to public data,
and the ruling is unfortunately not definitive.
It could still be appealed to the Supreme Court,
and that court might rule differently.
In any case, it only applies to America.
Ultimately,
the best outcome would be for governments to pass laws that clearly
permit web scraping in appropriate situations.
Until that happens, just be careful.
Automated web browsing then is ethical as long as you use the data
ethically. It's unlikely that web scraping will land you in jail.
In some cases, you can receive a cease and desist letter,
even get sued, so be careful, especially if you're scraping for profit,
scraping nonpublic data, or scraping personal information.
And for the best advice, consult an attorney.