Using OpenCV to Make A Video Game Bot

Motivation

In the past few months, I have been spending lots of my free time playing a video game called World of Warcraft.

I find myself having lots of trouble keeping track of all the different abilities’ cooldowns. And find this process very repetitive…

Thus I decided to make a bot to automate this process, so that I can truely enjoy the graphics of the game instead of looking at ability icons all the time.

Sample Bot

Development Process: basic framework

Reading Game State

WoW Addon: Weakauras

An in-game plugin/addon that can extract nearly all the information from the game to your UI in the form of text, icons and sound. I use Weakauras to extract following information from the game:

1. ability cooldowns (when can I use this ability)
2. whether I am in attack range, how many enemies are in attack-range
3. certain buff, or certain debuff on enemy (buff/debuff are lasting spell that benefit/harm myself/the enemy)
4. …

OpenCV: Template Matching

Template Matching is a Computer Vision technique to match a template image in a large image. It is like finding [1, 2] in a large matrix. It is used to capture all the information shown by Weakauras.

Damage Rotation

I have python function that takes in all the information collected from OpenCV, and computes the next best ability to use. Next is to “press the button” via python-win32pi and python-win32con if

• character is not on GCD (cooldown between each ability use)
• character is in attack range
• character is not casting a spell already

I reference following websites to write this function:

• raidbots
• wowhead
• icyveins

Below are some images how a typical damage rotation looks like.

feature development

1. bot that runs basic dmg rotation trigger by using specific ability in game
2. can toggle the bot on-and-off in game using hot-keys
3. support tracking the last ability used and not use duplicate abilities in certain situations (for WoW class Monk)
4. support doing burst damage macros: using one hot key to cast multiple spells in a sequence
5. support more hot-keys to toggle different modes
   • mode that optimized for killing mobs
   • mode that optimized for killing dungeon bosses
6. support reading text info like health/mana with PaddleOCR,
   can also used to detect mob’s position, health