.NET Get website source using cookie and proxy

[.NET] Get website source using cookie and proxy - Printable Version

+- Hack Forums (https://hackforums.net)
+-- Forum: Tech (https://hackforums.net/forumdisplay.php?fid=444)
+--- Forum: Coding (https://hackforums.net/forumdisplay.php?fid=151)
+---- Forum: Software Development (https://hackforums.net/forumdisplay.php?fid=118)
+---- Thread: [.NET] Get website source using cookie and proxy (/showthread.php?tid=5105305)

Get website source using cookie and proxy - SIM-2362662 - Dec 23, 2015

So i basically need a code to get a website source.
It has to use cookies (because otherwise that website will return an error) and be able to use proxy.

I tried this code :

Code 
Private Function page_source(ByVal URL As String, proxy As String) As String

        Try

            Dim reader As StreamReader

            Dim Request As HttpWebRequest = HttpWebRequest.Create(URL)

            Request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14"

            Request.CookieContainer = CC

            If proxy.Length > 4 Then

                Dim myProxy As New WebProxy(proxy)

                Request.Proxy = myProxy

            End If

            Dim Response As HttpWebResponse = Request.GetResponse()

            reader = New StreamReader(Response.GetResponseStream())

            Return reader.ReadToEnd()

            Response.Close()

        Catch ex As Exception

            MsgBox(ex.Message)

        End Try

    End Function

This code works fine on most of the websites i tried it. But not on the website i need right now.
That website is : http://pastebin.com/mQqLKtes

* - Kraizy - Dec 23, 2015

What kind of error do you get?

* - SIM-2362662 - Dec 23, 2015

(Dec 23, 2015 08:09 AM)Kraizy Wrote: What kind of error do you get?

[Image: 1af83456b69cddf5208b66cbf56ce8b2.png]

* - Kraizy - Dec 23, 2015

Do you get this error when sending the GET request to amazon.com or do you POST something?

* - SIM-2362662 - Dec 23, 2015

(Dec 23, 2015 08:20 AM)Kraizy Wrote: Do you get this error when sending the GET request to amazon.com or do you POST something?

I'm not sending any POST data. I'm using the code I posted on this thread.

* - Kraizy - Dec 23, 2015

Well, I guess it works fine when not using any proxy, right? Think it has something to do with your proxy, maybe blacklisted or something else...where do you get your proxies from?

* - Stаlkеr - Dec 23, 2015

Just saying myhabit nearly uses more captchas than amazon ^^ and you a other url with for a cracker btw I'll have a look at it :)

* - SIM-2362662 - Dec 23, 2015

(Dec 23, 2015 08:24 AM)Kraizy Wrote: Well, I guess it works fine when not using any proxy, right? Think it has something to do with your proxy, maybe blacklisted or something else...where do you get your proxies from?

I'm not even using a proxy. this is the code i use :

Dim source As String = page_source(url, "")

(Dec 23, 2015 08:25 AM)๖ۣۜStalker Wrote: Just saying myhabit nearly uses more captchas than amazon ^^ and you a other url with for a cracker btw I'll have a look at it :)

Number of captchas doesnt really matter. because the cracker will use proxies. So it will try just a single time with each proxy.
Thank you for looking into it . Let me know if you find anything 🤝

* - Kraizy - Dec 23, 2015

Don't know..I don't have any problems printing the html source with those few lines:

PHP Code 


<?php 
Dim req As Net.HttpWebRequest = Net.HttpWebRequest.Create("https://www.amazon.com/ap/signin?openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&pageId=quarterdeck&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&clientContext=175-9903052-0801208&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.mode=checkid_setup&marketPlaceId=A39WRC2IB8YGEK&openid.assoc_handle=quarterdeck&openid.return_to=https%3A%2F%2Fwww.myhabit.com%2Fsignin&&siteState=http%3A%2F%2Fwww.myhabit.com%2Fhomepage%3Fhash%3D%26weblabs%3DGodsonGiveGet%2FC%3B")
        
        Dim myProxy As New Net.WebProxy("ip", port)
        req.Proxy = myProxy

        Dim res As Net.HttpWebResponse = req.GetResponse()
        Dim strm As New IO.StreamReader(res.GetResponseStream())
        Dim html As String = strm.ReadToEnd()

        Debug.Print(html) 

* - SIM-2362662 - Dec 23, 2015

(Dec 23, 2015 08:32 AM)Kraizy Wrote: Don't know..I don't have any problems printing the html source with those few lines:

Are you sure?
Because this is what i get :
Spoiler (Click to View)

EDIT : It was not working because of a HF typo bug. It works fine if i dont use any proxy. But it return an error if i try to use proxy. This error :
Spoiler (Click to View)

* - Kraizy - Dec 23, 2015

Well I think it has something to do with your proxies, maybe they are broke?

[Image: 2bf3253eec.png]

* - SIM-2362662 - Dec 23, 2015

(Dec 23, 2015 08:48 AM)Kraizy Wrote: Well I think it has something to do with your proxies, maybe they are broke?

Yep . I think that was the problem! Thank you so much!