Home Upgrade Search Memberlist Extras Hacker Tools Award Goals Help Wiki Contact

Python 

HideMyAss.com Proxy Scraper

Dirty code, again :)

I've used eventlet module to speed up the scraping progress. You could ofc use just for url in urls: urllib.urlopen("")

getProxies() returns a list containing dictionaries for each individual proxy on that page.

http://pastebin.com/apnDFm1i
Looking into eventlet now.

Interested in how the concurrency gives a performance boost.

Thanks for the pastebin,
(Jan 26, 2014 07:13 PM)K.ing Wrote:  Looking into eventlet now.

Interested in how the concurrency gives a performance boost.

Thanks for the pastebin,

Yeah well, eventlet is basically just threading but much easier and I definitely like it better than the normal way :P
wow, this is kick ass! good job!
I'll definitely be using this!
I was going to do this with BeautifulSoup, but it's honestly more efficient with regular expressions in my opinion.

hi
Thanks for the feedback :) Glad you like it.


Possibly Related Threads...
Thread / Author Replies Views Last Post
by greekdevSep 9, 2026 10:36 AM 2 126 Sep 9, 2026 10:36 AM
Last Post: greekdev
[Python] by frogeyDec 11, 2017 01:05 AM 1 296 Dec 11, 2017 01:05 AM
Last Post: dreadpixel
[C#] by CharculOct 6, 2017 08:49 AM 16 476 Oct 6, 2017 08:49 AM
Last Post: Ninja