Auto Maple
Auto Maple is an intelligent Python AI that plays MapleStory, a 2D side-scrolling MMORPG, using simulated key presses, TensorFlow machine learning, OpenCV template matching, and other computer vision techniques.
Community-created resources, such as command books for each class and routines for each map, can be found in the resources repository.
Minimap
Auto Maple uses OpenCV template matching to determine the bounds of the minimap as well as the various elements within it, allowing it to accurately track the player's in-game position. If record_layout is set to True, Auto Maple will record the player's previous positions in a quadtree-based Layout object, which is periodically saved to a file in the "layouts" directory. Every time a new routine is loaded, its corresponding layout file, if it exists, will also be loaded. This Layout object uses the A* search algorithm on its stored points to calculate the shortest path from the player to any target location, which can dramatically improve the accuracy and speed at which routines are executed.
Command Books
The above video shows Auto Maple consistently performing a mechanically advanced ability combination.
Designed with modularity in mind, Auto Maple can operate any character in the game as long as it is provided with a list of in-game actions, or a "command book". A command book is a Python file that contains multiple classes, one for each in-game ability, that tells the program what keys it should press and when to press them. Once a command book is imported, its classes are automatically compiled into a dictionary that Auto Maple can then use to interpret commands within routines. Commands have access to all of Auto Maple's global variables, which can allow them to actively change their behavior based on the player's position and the state of the game.
Routines
Click [here](https://github.com/tanjeffreyz02/auto-maple/blob/f13d87c98e9344e0a4fa5c6f85ffb7e66860afc0/routines/dcup2.csv) to view the entire routine.
A routine is a user-created CSV file that tells Auto Maple where to move and what commands to use at each location. A custom compiler within Auto Maple parses through the selected routine and converts it into a list of Component objects that can then be executed by the program. An error message is printed for every line that contains invalid parameters, and those lines are ignored during the conversion.
Below is a summary of the most commonly used routine components:
Point stores the commands directly below it and will execute them in that order once the character is within move_tolerance of the specified location. There are also a couple optional keyword arguments:
adjust fine-tunes the character's position to be within adjust_tolerance of the target location before executing any commands.
frequency tells the Point how often to execute. If set to N, this Point will execute once every N iterations.
skip tells the Point whether to run on the first iteration or not. If set to True and frequency is N, this Point will execute on the N-1th iteration.
Label acts as a reference point that can help organize the routine into sections as well as create loops.
Jump jumps to the given label from anywhere in the routine.
Setting updates the specified setting to the given value. It can be placed anywhere in the routine, so different parts of the same routine can have different settings. All editable settings can be found at the bottom of [settings.py](https://github.com/tanjeffreyz02/auto-maple/blob/v2/settings.py).
Runes
Auto Maple has the ability to automatically solve "runes", or in-game arrow key puzzles. It first uses OpenCV's color filtration and Canny edge detection algorithms to isolate the arrow keys and reduce as much background noise as possible. Then, it runs multiple inferences on the preprocessed frames using a custom-trained TensorFlow model until two inferences agree. Because of this preprocessing, Auto Maple is extremely accurate at solving runes in all kinds of (often colorful and chaotic) environments.
Video Demonstration
Click below to watch the full video
Setup
Download and install [Python3](https://www.python.org/downloads/).
Download and install the latest version of [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads).
Download and install [Git](https://git-scm.com/download/win).
Download and unzip the latest [Auto Maple release](https://github.com/tanjeffreyz02/auto-maple/releases).
Download the [TensorFlow model](https://drive.google.com/drive/folders/1SPdTNF4KZczoWyWTgfyTBRvLvy7WSGpu?usp=sharing) and unzip the "models" folder into Auto Maple's "assets" directory.
Inside Auto Maple's main directory, open a command prompt and run:
python -m pip install -r requirements.txt
Lastly, create a desktop shortcut by running:
python setup.py
This shortcut uses absolute paths, so feel free to move it wherever you want. However, if you move Auto Maple's main directory, you will need to run python setup.py again to generate a new shortcut. To keep the command prompt open after Auto Maple closes, run the above command with the --stay flag.