Using OpenCV to Make A Video Game Bot


In the past few months, I have been spending lots of my free time playing a video game called World of Warcraft.

I find myself having lots of trouble keeping track of all the different abilities’ cooldowns. And find this process very repetitive…

Thus I decided to make a bot to automate this process, so that I can truely enjoy the graphics of the game instead of looking at ability icons all the time.

Sample Bot

Alt Text

Development Process: basic framework

Reading Game State

WoW Addon: Weakauras

An in-game plugin/addon that can extract nearly all the information from the game to your UI in the form of text, icons and sound. I use Weakauras to extract following information from the game:

  1. ability cooldowns (when can I use this ability)
  2. whether I am in attack range, how many enemies are in attack-range
  3. certain buff, or certain debuff on enemy (buff/debuff are lasting spell that benefit/harm myself/the enemy)

OpenCV: Template Matching

Template Matching is a Computer Vision technique to match a template image in a large image. It is like finding [1, 2] in a large matrix. It is used to capture all the information shown by Weakauras.

Damage Rotation

I have python function that takes in all the information collected from OpenCV, and computes the next best ability to use. Next is to “press the button” via python-win32pi and python-win32con if

I reference following websites to write this function:

Below are some images how a typical damage rotation looks like.
image tooltip here image tooltip here image tooltip here

feature development

  1. bot that runs basic dmg rotation trigger by using specific ability in game
  2. can toggle the bot on-and-off in game using hot-keys
  3. support tracking the last ability used and not use duplicate abilities in certain situations (for WoW class Monk)
  4. support doing burst damage macros: using one hot key to cast multiple spells in a sequence
  5. support more hot-keys to toggle different modes
    • mode that optimized for killing mobs
    • mode that optimized for killing dungeon bosses
  6. support reading text info like health/mana with PaddleOCR,
    can also used to detect mob’s position, health
Table of Contents