Using OpenCV to Make A Video Game Bot
In the past few months, I have been spending lots of my free time playing a video game called World of Warcraft.
I find myself having lots of trouble keeping track of all the different abilities’ cooldowns. And find this process very repetitive…
Thus I decided to make a bot to automate this process, so that I can truely enjoy the graphics of the game instead of looking at ability icons all the time.
Development Process: basic framework
Reading Game State
WoW Addon: Weakauras
An in-game plugin/addon that can extract nearly all the information from the game to your UI in the form of text, icons and sound. I use Weakauras to extract following information from the game:
- ability cooldowns (when can I use this ability)
- whether I am in attack range, how many enemies are in attack-range
- certain buff, or certain debuff on enemy (buff/debuff are lasting spell that benefit/harm myself/the enemy)
OpenCV: Template Matching
Template Matching is a Computer Vision technique to match a template image in a large image. It is like finding [1, 2] in a large matrix. It is used to capture all the information shown by Weakauras.
I have python function that takes in all the information collected from OpenCV, and computes the next best ability to use. Next is to “press the button” via
- character is not on GCD (cooldown between each ability use)
- character is in attack range
- character is not casting a spell already
I reference following websites to write this function:
Below are some images how a typical damage rotation looks like.
- bot that runs basic dmg rotation trigger by using specific ability in game
- can toggle the bot on-and-off in game using hot-keys
- support tracking the last ability used and not use duplicate abilities in certain situations (for WoW class Monk)
- support doing burst damage macros: using one hot key to cast multiple spells in a sequence
- support more hot-keys to toggle different modes
- mode that optimized for killing mobs
- mode that optimized for killing dungeon bosses
- support reading text info like health/mana with PaddleOCR,
can also used to detect mob’s position, health