+- Hack Forums (https://hackforums.net)
+-- Forum: Tech (https://hackforums.net/forumdisplay.php?fid=444)
+--- Forum: Coding (https://hackforums.net/forumdisplay.php?fid=151)
+---- Forum: Software Development (https://hackforums.net/forumdisplay.php?fid=118)
+---- Thread: [Python] Image Recognition/Clicking an Image with Mouse (/showthread.php?tid=5957774)
Image Recognition/Clicking an Image with Mouse - District - Apr 7, 2019
Hi HF. I'm a beginner and I'm trying to learn how to use Python to recognize an image, so that later in the script I can use mouse/keyboard events on said image. In my head it sounds simple but so far I'm struggling to find an answer. All I want the script to do is recognize the image(s) on the screen and allow me to execute commands when it sees them, like clicking it. There may be situations where there are multiple of the same image on the screen and it will need to click them in order.
I've been trying out so many modules/libraries and I can't seem to find an answer. I only have a knowledge of the basics but even reading the docs on certain module features hasn't been helping me, as the information given seems to be very basic.
I've been trying Pytesseract and even when running the code below I get no errors but it prints a blank line. No information whatsoever. I don't know if I'm doing something wrong or if I'll require something else to work with it. The whole point of this is that I want to learn how to make Python recognize the image, and I want to understand what each command/part of the code does.
Code
from PIL import Image
import pytesseract
pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract"
print(pytesseract.image_to_string(Image.open('letter_a.png')))Can anyone give me some information or recommend a different approach? Talking with someone 1:1 who has done this before would be a big help too. I'd even be willing to pay someone for knowledge on this.
RE: Image Recognition/Clicking an Image with Mouse - loomski - Apr 7, 2019
Have you tried using pyautogui to do this? It has screenshot functions, mouse, and keyboard functions as well.
Or, are you set on using pytesseract?
I did something similar to this some time ago, having to get coordinates of pixels which later turned into a screenshot of an area, and pressing keyboard buttons in certain conditions.
Look into that module maybe
RE: Image Recognition/Clicking an Image with Mouse - lockhead - Apr 7, 2019
Add this to your code:
Code
str_img = pytesseract.image_to_string(Image.open('letter_a.png'))
print(type(str_img))
print(len(str_img))That's my best guess.
Make sure the image is in the correct folder.
Try it with a different image.
RE: Image Recognition/Clicking an Image with Mouse - District - Apr 7, 2019
(Apr 7, 2019 02:09 AM)lockhead Wrote: Add this to your code:
Since it prints a blank line it probably couldn't find the image (although that probably should throw an error), couldn't read the image or the image was empty.Codestr_img = pytesseract.image_to_string(Image.open('letter_a.png'))
print(type(str_img))
print(len(str_img))
That's my best guess.
Make sure the image is in the correct folder.
Try it with a different image.
The path to the image is fine. I tried what you said and at least this time it gave me some info, not sure if it is the info I need but it returned:
Code
<class 'str'>
0Edit: Tried another image which is a jpg and it threw the same information.
RE: Image Recognition/Clicking an Image with Mouse - lockhead - Apr 7, 2019
(Apr 7, 2019 02:25 AM)I Am Phil Wrote: The path to the image is fine. I tried what you said and at least this time it gave me some info, not sure if it is the info I need but it returned:This means you have an empty string. Check the documentation, maybe it'll tell you why or when an empty string is returned.
Code<class 'str'>
0
Edit: Tried another image which is a jpg and it threw the same information.