[permutate]

Welcome!
Preface: If you want to get into serious OSRS Bot development, check out Java.
This thread's purpose is to educate beginners on introductory mouse and keyboard automation in Old School RuneScape using the Python module PyAutoGUI.
I think PyAutoGUI is a fun and easy module to play around with, and OSRS is a great sandbox.

Most beginner to intermediate python projects are boring. How many calculators, Tic-Tac-Toe solvers, or number guessing games do we need?
This is mainly targeted at Python beginners who want to take on a non-vanilla project that is both fun and rewarding.

In this tutorial, we'll be using the PyAutoGUI module to automate mouse and keyboard functions.
More specifically, we'll be writing a simple script that clicks on two inventory items, presses space, and waits.
You can use these fundamentals to then automate fletching logs, making wine, making unfinished potions, etc.

Introduction
I wrote PyAutoGUI scripts mainly to help me clean herbs and make unfinished potions for a profit. I was averaging 500-600k/hr on one account just cleaning grimy herbs and making unfinished potions out of them and then selling them for profit.
However, my account did get banned for two days because I left the script running for 6-10 hours, on multiple consecutive days. If you plan to use this on your main account, be smart, since this is a very simple script. I would recommend taking breaks and actually playing the account legit for a period of time between botting sessions, and use common sense to avoid bans. Basic scripts like this will have a high chance of ban. However, well-written custom scripts can reduce ban-rate, as custom scripts are harder to detect.

Requirements

• Basic/Beginner's knowledge of Python
  


• Having read Automate the Boring Stuff with Python, is a plus (see chapter 20).
  


• Latest Python version. Python2 works, but I will be using Python3 for this tutorial.
  


• An OSRS account you don't care about is recommended.
  

PyAutoGUI for OSRS Botting

We'll be using PyAutoGUI, as the API is simple and easy to learn.
As you advance onwards, you can utilize the screenshot functions  to locate saved images within the screen.
However, for the sake of the scope of this tutorial, we will only use the mouse control functions and the keyboard control functions.

AutoHotKey vs PyAutoGUI vs pynput
For those of you already in the know, you know that there are thousands of AHK scripts already out there for OSRS. Why should we bother with PyAutoGUI?

1. It's a Python tutorial!
   


2. AutoHotKey's syntax can be confusing to beginners. Python has syntax that is easier to read, more documentation, and more support.
   


3. Us MacOS/Linux users can't use AHK.
   


4. pynput's functionality is almost identical to PyAutoGUI. Use whichever you prefer. In my own scripts I use them both. PyAutoGUI is better at automation, and pynput is better with monitoring/listening to mouse and keyboard events.
   

Installing PyAutoGUI
Go through the PyAutoGUI installation guide and install the PyAutoGUI module.


Starting
Keep the pyautogui cheatsheet close.
Let's write our first, and most important script: mouseInfo.py

> mouseInfo.py

Code

import pyautogui

pyautogui.mouseInfo()

That's it. When ran, mouseInfo.py displays a GUI that contains the current XY coordinates of the mouse cursor, with a hotkey option to log where your cursor is at (F6). This tool is very useful for planning out automation that involves clicking on the screen at specific coordinates. We'll use this script later to plan out the locations we want the mouse to click at.

Next, we'll want to create a new python file for our main script. I'm naming mine unfinished.py since I'm writing a script to mix unfinished potions.

> unfinished.py

Code

import pyautogui
import sys
import time
import random
import math
from random import randint

pyautogui.FAILSAFE = True

• When the program is running, and the user moves their cursor to the top-left of their screen with their mouse, the program will raise a FailSafeException and abort.
  

Logging XY-Values and Planning Automation
Now, open up OSRS and log the XY coordinates for each inventory space (28). You will get (4) unique X-values and (7) unique Y-values.

• Log the values with the F6 hotkey using the mouseInfo.py GUI, or with the default 3 sec. button delay.
  


• Save the log as a TXT file
  


• You don't really need to log all 28 XY-values. After the first few, you'll see that the inventory is a grid and you can use math to figure out the rest. The offsets for X and Y are roughly 40-50px on my monitor.
  

IMPORTANT: Because this script relies on XY coordinates (absolute positions), ensure that your OSRS client window is at the same fixed location whenever you want to run the script.

With your logged TXT file, fill this with your recorded XY values and append it to unfinished.py:
    - The Top Left inventory slot is (x1, y1), and the Bottom Right is (x4, y7)

> unfinished.py

Code

def makePot:
    # the 0th index is pre-filled with a Zero so that we can have invX[1] correspond to x1.
    invX = [0, x1, x2, x3, x4]
    invY = [0, y1, y2, y3, y4, y5, y6, y7]

With the initial logging done, we can now plan out the steps we want executed.
For my example, I will be automating making unfinished potions, but you can be creative and make this into anything that you'd like. (Fletching, Alching, CBalls, etc).

The sequence of events will look something like this:

Code

Click on herb (x1, y1)
Click on Vial of Water (x3, y4) (the 14th inventory slot)
Press Space
Wait for interaction to complete.

Tweening / Easing Functions DOCS
Tweening makes mouse movements fancy. You can skip this section and moveTo functions if you don't care about that.
A tweening function dictates the speed and movement of the mouse as it reaches its destination. See the DOCS for more information.

Writing the script
Then, we convert it to python using Mouse Control Functions and Keyboard Control Functions
> unfinished.py

Code

def makePot:
    # the 0th index is pre-filled with a Zero so that we can have invX[1] correspond to x1.
    invX = [0, x1, x2, x3, x4]
    invY = [0, y1, y2, y3, y4, y5, y6, y7]

    # Amount of time we want each moveTo action to take (in seconds). Randomize this speed to mimic human-like behavior.
    # Change this if you don't want each moveTo action to take 2-3 seconds.
    mouseSpeed = 1 + random.uniform(1, 2)

    # pyautogui.moveTo(x, y, mouseSpeed, tweeningFunction)
    # pyautogui.click(x, y)
 
    # Step 1: move to and click on herb (x1, y1)
    pyautogui.moveTo(invX[1], invY[1], mouseSpeed, pyautogui.easeInOutQuad)
    pyautogui.click(invX[1], invY[1])

    # Step 2: move to and click vial of water (x3, y4)
    pyautogui.moveTo(invX[3], invY[4], mouseSpeed, pyautogui.easeInOutQuad)
    pyautogui.click(invX[3], invX[4])

    # Step 3: press space
    pyautogui.press('space')

    # Step 4: wait for interaction to complete
    # It takes around 9 seconds to finish mixing unfinished potions.
    time.sleep(9)

That's it! You've just automated making unfinished potions.

Now, all that's left is for you to write the other tasks:

Code

# you didn't think I'd give you the whole thing, right?
def bank:
    # bank inventory
    # withdraw herbs
    # withdraw vials of water
    # exit bank

def execute:
    makePot()
    bank()

loop:
    execute()

... And that's it! You've just created a very simple automation script for OSRS using Python. While it is not necessarily the most complex or useful at its current state, with some love and attention, it becomes a solid crafting script.

Low hanging fruit to tackle next

• Random delays: the very first thing you might notice when you run your script is that without some delay between consecutive actions, some actions are executed too fast for your OSRS client to catch up. You'll want to add extra random delays sandwiched between lines of code that will execute an action. ex: the time between clicking the second ingredient and the time it takes for OSRS to pop-up the Make-x prompt.
  


• Random offsets for each click: real players won't click on the same spot multiple times. Add a random offset of a few pixels to each action.
  


• Take a break: real players won't play for hours without stopping. Implement a case where the bot will 'take a break'.
  


• Hotkeys: using pynput, we are able to listen to certain keyboard events. Use this to your advantage and create a hotkey that will start your main loop when pressed. Also add a hotkey to abort the script if anything goes wrong.
  


• Screenshot functions: relying on absolute positioning sucks if you want to seriously bot. Screenshot functions are the next step you'll want to take when moving away from absolute positioning reliant scripts. Unfortunately, the Pillow module required for Screenshot functions isn't supported in Python3+, so you'll have to use Python2 if you want to use screenshot functions.
  


• VM: Setting this up in a VM environment is also a smart move if you don't want to be mouse restricted.
  

Note: Mouse Restriction
Please note that when PyAutoGUI is executing mouse and keyboard actions, you won't be able to control your computer for that specific amount of time.

For example: if you have a herb cleaning script that is written to click on each herb individually, PyAutoGUI takes over your mouse and performs those actions.
You won't be able to easily move your cursor easily during this period of time. Your actions will fight with PyAutoGUI's and it gets messy very quickly. But, if you have a cball script that isn't click intensive, you really only need to worry about the program clicking once every 3 minutes. Running this script in a VM solves this issue.

If you've read the whole thing, thanks for reading!
This is the first guide I've written here. I'm not a Python expert so if there are any improvements to be made, feel free to critique me below!

hit the 12k char limit, so start convo if you wanna talk :)

[tsuki]

Thanks for sharing. It's rare to see HQ programming threads anymore.

I used PyAutoGUI years ago to code an anti-AFK splashing script that I'd use like 12 hours a day. It would randomly alt tab into the game window every X minutes, click some buttons, and alt tab back out. So I tried to imitate the actions of a legitimate player as much as possible who's not really paying attention and only occasionally tabs in to avoid getting logged out. Though I'd be scared to use it for anything else like cleaning herbs or alching, which sounds a lot riskier.

[permutate]

Thanks for sharing. It's rare to see HQ programming threads anymore.

I used PyAutoGUI years ago to code an anti-AFK splashing script that I'd use like 12 hours a day. It would randomly alt tab into the game window every X minutes, click some buttons, and alt tab back out. So I tried to imitate the actions of a legitimate player as much as possible who's not really paying attention and only occasionally tabs in to avoid getting logged out. Though I'd be scared to use it for anything else like cleaning herbs or alching, which sounds a lot riskier.

I'm glad you enjoyed the thread! And thank you for mentioning that, I totally forgot that splashing is a thing in runescape, an anti-logout script for training mage is super helpful and also has very few actions to perform, and perfect for any beginners trying out PyAutoGUI.

[G.P.D.]

Thanks for this thread OP !
I don't play RS but PyAutoGUI can be used for a lot of other things.
I'm curious about how the integration of the screenshot function could be done. Is it an AI that search for cases ?

[Harley Quinn]

This is really interesting! I am looking forward to getting into this library.

Thanks for the tutorial and information! Will update when i have dabbled

[tux1]

HQ thread, thanks for sharing.
I hope you continue to share more with the community

[Lindros]

This is really interesting. I no longer play OSRS cause I just end up turning into a gambling addict but thanks for the share.

[Ovidiu]

Bruh great post, I've been thinking of learning automation with python by doing an OSRS project.

[permutate]

This is really interesting. I no longer play OSRS cause I just end up turning into a gambling addict but thanks for the share.

Hahaha that's exactly what I've become. After discovering how much money I can make in the sand casino and other gambling sites, regular accounts lost their sparkle, and I pretty much only bot or gamble on them.

I occasionally play on my ironman account since I can't cheat and it's been a lot of fun - time consuming but it brings back the classic feel of playing runescape.

Thanks for this thread OP !
I don't play RS but PyAutoGUI can be used for a lot of other things.
I'm curious about how the integration of the screenshot function could be done. Is it an AI that search for cases ?

I've always wondered what other people used PyAutoGUI for, as OSRS was the first thing that popped into my mind when I found this Python Automation module.

Regarding the screenshot function, it's not anything as complex as an AI, but rather it scans the screen for any matches to the saved image, and returns the coordinates if it locates any instances of the image on the screen. It's finicky and not that fast, but not necessarily slow, so there isn't any good reason to use PyAutoGUI's screenshot functions unless you really have to.

----------

Thanks for the kind replies everyone! I noticed a significant overlap between players of OSRS and members of this community, so I figured I would share this information that I hadn't found anywhere else. As a new member of this board, I didn't know how well this thread would be received, but I'm glad I can contribute!

If you have any questions with Python or PyAutoGUI, feel free to start a convo with me. I'm much more responsive there. However do note that I only have basic experience with PyAutoGUI, I picked it up mess around with Python as I am primarily a Front End dev.