+- Hack Forums (https://hackforums.net)
+-- Forum: Tech (https://hackforums.net/forumdisplay.php?fid=444)
+--- Forum: Coding (https://hackforums.net/forumdisplay.php?fid=151)
+---- Forum: Software Development (https://hackforums.net/forumdisplay.php?fid=118)
+---- Thread: [Python] Bypass Cloudflare antibot protection for scraping (/showthread.php?tid=6040967)
Bypass Cloudflare antibot protection for scraping - UKevin - Jan 14, 2020
Is there any framework out there to do so?
RE: Bypass Cloudflare antibot protection for scraping - lnvisible - Jan 14, 2020
I would checkout puppeteer-extra, it's a wrapper that has addons that makes it harder for websites to detect bots. I haven't used puppeteer for web scraping, but for other various tasks.
RE: Bypass Cloudflare antibot protection for scraping - AeroBlue - Jan 14, 2020
I would highly suggest using something such as cloudscraper, I have used it before and it bypasses Cloudflare on any website using javascript. It also acts pretty much like normal requests as well making it easy to use.
RE: Bypass Cloudflare antibot protection for scraping - Intense - Jan 15, 2020
(Jan 14, 2020 10:11 PM)Aero Blue Wrote: I would highly suggest using something such as cloudscraper, I have used it before and it bypasses Cloudflare on any website using javascript. It also acts pretty much like normal requests as well making it easy to use.
I second this. Cloudscraper has been a very useful and effective module for me several times.
RE: Bypass Cloudflare antibot protection for scraping - UKevin - Jan 15, 2020
Thx for the advices, anything with python??
RE: Bypass Cloudflare antibot protection for scraping - SIM-1866423 - Jan 17, 2020
(Jan 14, 2020 10:11 PM)Aero Blue Wrote: I would highly suggest using something such as cloudscraper, I have used it before and it bypasses Cloudflare on any website using javascript. It also acts pretty much like normal requests as well making it easy to use.
https://github.com/VeNoMouS/cloudscraper/commit/0025582ac76a9dfa3cf69bce401e9c7a149afd21
seems like its updated good news !