Is there any framework out there to do so?
|
Python
Bypass Cloudflare antibot protection for scraping |
I would checkout puppeteer-extra, it's a wrapper that has addons that makes it harder for websites to detect bots. I haven't used puppeteer for web scraping, but for other various tasks.
![[Image: ZEve4.png]](https://i.imgur.com/ZEve4.png)
Jan 14, 2020 10:11 PM
(This post was last modified: Jan 14, 2020 10:11 PM by AeroBlue.)
I would highly suggest using something such as cloudscraper, I have used it before and it bypasses Cloudflare on any website using javascript. It also acts pretty much like normal requests as well making it easy to use.
 (Jan 14, 2020 10:11 PM)AeroBlue Wrote: I would highly suggest using something such as cloudscraper, I have used it before and it bypasses Cloudflare on any website using javascript. It also acts pretty much like normal requests as well making it easy to use. I second this. Cloudscraper has been a very useful and effective module for me several times.
Jan 15, 2020 07:35 AM
(This post was last modified: Jan 15, 2020 10:22 AM by UKevin.)
Thx for the advices, anything with python??
 (Jan 14, 2020 10:11 PM)AeroBlue Wrote: I would highly suggest using something such as cloudscraper, I have used it before and it bypasses Cloudflare on any website using javascript. It also acts pretty much like normal requests as well making it easy to use. https://github.com/VeNoMouS/cloudscraper...7a149afd21 seems like its updated good news ! |
|
|
