Hack Forums
[.NET] Scan screen for specific pixel color, move mouse to it, and click it. - Printable Version

+- Hack Forums (https://hackforums.net)
+-- Forum: Tech (https://hackforums.net/forumdisplay.php?fid=444)
+--- Forum: Coding (https://hackforums.net/forumdisplay.php?fid=151)
+---- Forum: Software Development (https://hackforums.net/forumdisplay.php?fid=118)
+---- Thread: [.NET] Scan screen for specific pixel color, move mouse to it, and click it. (/showthread.php?tid=4080620)



Scan screen for specific pixel color, move mouse to it, and click it. - ⁠Eloquence - Feb 24, 2014

I am working on something were I need to find a specific pixel color and move the mouse to it. After the mouse is there I need to click it. And issue is there are numerous of the same pixel and I need to be able to have it click all of them. Any help with this would be greatly appreciated. Thanks!


* - derpin0n - Feb 24, 2014

Try using a for each loop.

http://msdn.microsoft.com/en-us/library/5ebk1751.aspx


* - ⁠Eloquence - Feb 24, 2014

(Feb 24, 2014 12:08 PM)derpin0n Wrote:  Try using a for each loop.

http://msdn.microsoft.com/en-us/library/5ebk1751.aspx

Okay, now how do I go about finding the pixel and moving the mouse to it?


* - derpin0n - Feb 24, 2014

Well, as I understood from your thread, you're saying you aleardy have that.

Anyway, you should take a look at the following APIs:
http://msdn.microsoft.com/en-us/library/windows/desktop/dd144909(v=vs.85).aspx
http://msdn.microsoft.com/en-us/library/windows/desktop/ms648394(v=vs.85).aspx


* - ⁠Eloquence - Feb 24, 2014

(Feb 24, 2014 12:52 PM)derpin0n Wrote:  Well, as I understood from your thread, you're saying you aleardy have that.

Anyway, you should take a look at the following APIs:
http://msdn.microsoft.com/en-us/library/windows/desktop/dd144909(v=vs.85).aspx
http://msdn.microsoft.com/en-us/library/windows/desktop/ms648394(v=vs.85).aspx

Those are for C++. Could you find it for vb I have been looking since you posted this.


* - Menalix - Feb 24, 2014

(Feb 24, 2014 01:21 PM)Eurɘka Wrote:  Those are for C++. Could you find it for vb I have been looking since you posted this.
Those are for all programming languages.
But the example syntax is written in c++


* - Hawk HF - Feb 24, 2014

I know that you can get the color of a pixel from a bitmap.

Look up a tutorial on how to get a screen capture, since I know there's a ton of them. Then use a loop to go through each pixel on the screen in search of your color.

Hope this helps.


* - marvinn - Feb 25, 2014

Create a bitmap from the screen (A screenshot). Use the bitmap function GetPixel(x,y) (http://msdn.microsoft.com/en-us/library/system.drawing.bitmap.getpixel(v=vs.110).aspx) and make a for loop, for (int w = 0; w < bitmap.Width; w++) to scan the whole bitmap. Hope you understand it :)


* - BlackCap - Feb 25, 2014

First of all you would need to get a picture of the area where your pixel are. The easiest way to do this would be to take a screenshot, but, it is also possible to make the excact windows / control that you are searching render itself on to a bitmap using eighter of the bitblt or printwindows api calls.
For the sake of simplicity I'm just going to take a screenshot in this example, but I would strongly urge you to use bitblt unless you're working with a fullscreen window.

PHP Code

<?php 
Function TakeScreenshot() As Bitmap
    Dim B 
As New Bitmap(My.Computer.Screen.Bounds.Width, My.Computer.Screen.Bounds.Height)
    
Dim G = Graphics.FromImage(B)

    
G.CopyFromScreen(My.Computer.Screen.Bounds.X, My.Computer.Screen.Bounds.Y, 0, 0, My.Computer.Screen.Bounds.Size)
    
G.Dispose()

    Return 
B
End 
Function 

Then you have to figure out where in the picture the color you seek are. The easiest way to go about this would be the GetPixel function in the graphics class. However, each time you use that method it will load the entire image into the memory, hand you that one pixel you asked for, and then burn the rest of the image; this is really really slow, so unless you really only need one pixel, you should load the image into memory youself. You can do so with the LockBits function.

NOT LIKE THIS:
PHP Code

<?php 
Dim s 
= TakeScreenshot()

For 
y = 0 To s.Height - 1
    
For x = 0 To s.Width - 1
      Dim pixel 
= s.GetPixel(x, y)
      
' ...
    Next
Next 

But like this:
PHP Code

<?php 
Iterator 
Function FindPixels(b As Bitmap, c As Color) As IEnumerable(Of Point)
    
Dim d = b.LockBits(New Rectangle(0, 0, b.Width, b.Height), Imaging.ImageLockMode.ReadOnly, b.PixelFormat)
    
Dim buf(d.Height * d.Stride - 1) As Byte

    
' Copy the bytes to memory (buf)
    Runtime.InteropServices.Marshal.Copy(d.Scan0, buf, 0, buf.Count)

    ' 
Loop trought all the bits in the image
    
For i = 0 To buf.Length - 1 Step 4 ' Each pixel contains four bytes (A, R, G, B)
      If c.A <> buf(i + 3) Then Continue For ' 
You can remove this as your picture doesn't contain opacity
      If c.R <> buf(i + 2) Then Continue For
      If c.G <> buf(i + 1) Then Continue For
      If c.B <> buf(i + 0) Then Continue For

      Dim ya = i / d.Stride
      Dim y = Math.Floor(ya)
      Dim x = (ya - y) * d.Width

      Yield New Point(CInt(x), Math.Floor(y))
    Next

    ' 
Finaly unlock the image 
    b
.UnlockBits(d)
End Function 

So at this point you have a list of points which needs to be clicked, and you just need a way to click them. The easiest thing to do here is to just "click"; however, it is also possible to just send a click message with API calls. The bitblt method can screenshot windows even if they are not visible on the screen, and nor does the thing that you click have to be visible on the screen for you to send it a click message.
However, if you're working with an application that are visible on the screen at all times, actually moving the cursor to a location and clicking should do just fine.

PHP Code

<?php 
Private Declare Sub mouse_event Lib "user32" (dwFlags%, dX%, dY%, cButtons%, dwExtraInfo As IntPtr)

Public Enum 
MouseEventFlags
    Move 
= 1 << 0
    LeftDown 
= 1 << 1
    LeftUp 
= 1 << 2
    RightDown 
= 1 << 3
    RightUp 
= 1 << 4
    MiddleDown 
= 1 << 5
    MiddleUp 
= 1 << 6
    ABSOLUTE 
= 1 << 15 ' Is it just me who think it's wierd that they didn't put this at 2^7 so that it would fit on a byte?
End Enum

Sub LeftClick()
    mouse_event(MouseEventFlags.LEFTDOWN Or MouseEventFlags.LEFTUP, 0, 0, 0, 0)
End Sub 

And then, finaly, to loop trought all the pixels on the screen and click a specific color:
PHP Code

<?php 
For Each p In FindPixels(TakeScreenshot, Color.Green)
    
Cursor.Position = p
    LeftClick
()
Next 



* - Onin42 - Feb 25, 2014

Hi if you need a source i have the perfect source for you, i created an roulette auto click. Arcade game betfair.

here is the code:

Code
Public Function ObserveColor(ByVal Pos As Point, ByVal C As Color, Optional Trys As Integer = 5) As Boolean
      Cursor.Position = Pos
      For i As Integer = 0 To Trys
      G.CopyFromScreen(Cursor.Position, Point.Empty, Bmp.Size)
      If Bmp.GetPixel(0, 0).Equals(C) Then
            Return True
      End If
      System.Threading.Thread.Sleep(150)
      Next
      Return False
    End Function

Code
Public Declare Auto Function SetCursorPos Lib "User32.dll" (ByVal X As Integer, ByVal Y As Integer) As Long
    Public Declare Auto Function GetCursorPos Lib "User32.dll" (ByRef lpPoint As Point) As Long
    Public Declare Sub Mouse_Event Lib "user32" Alias "mouse_event" (ByVal dwFlags As Integer, ByVal dx As Integer, ByVal dy As Integer, ByVal cButtons As Integer, ByVal dwExtraInfo As Integer)
    Public Const MOUSEEVENTF_LEFTDOWN = &H2
    Public Const MOUSEEVENTF_LEFTUP = &H4
    Public Const MOUSEEVENTF_MIDDLEDOWN = &H20
    Public Const MOUSEEVENTF_MIDDLEUP = &H40
    Public Const MOUSEEVENTF_RIGHTDOWN = &H8
    Public Const MOUSEEVENTF_RIGHTUP = &H10
    Public Const MOUSEEVENTF_MOVE = &H1

    Public Sub MouseClick(ByVal MouseButton As MouseButtons, ByVal MousePos As Point)
      Cursor.Position = MousePos

      Select Case MouseButton

      Case MouseButtons.Left
            Mouse_Event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0)
            Mouse_Event(MOUSEEVENTF_LEFTUP, 0, 0, 0, 0)

      Case MouseButtons.Right
            Mouse_Event(MOUSEEVENTF_RIGHTDOWN, 0, 0, 0, 0)
            Mouse_Event(MOUSEEVENTF_RIGHTUP, 0, 0, 0, 0)

      End Select
    End Sub