Pathfinding Benchmarks

This page is part of Nathan Sturtevant's Moving AI Lab at the University of Denver.

There is a wide body of researchers who use gridworld domains as benchmarks. The goal of this page is to collect benchmark problems and maps that can be broadly used and referenced. A technical report describing the maps is currently in preparation. But, the data is being made available to help improve the consistency of experiments when it comes to gridworlds.

BioWare deserves strong recognition for giving permission for their maps to be distributed for research purposes. We encourage other game companies to follow BioWare's lead and make map test sets available for research and testing purposes.

Other maps sets are available, but explicit permission for redistribution has not been acquired. These will be removed upon request.

Each benchmark set has a number of metrics associated with it. A short paper will be published shortly. These benchmarks can be cited as:

@article{sturtevant2012benchmarks,
  title={Benchmarks for Grid-Based Pathfinding},
  author={Sturtevant, N.},
  journal={Transactions on Computational Intelligence and AI in Games},
  volume={4},
  number={2},
  pages={144 -- 148},
  year={2012},
  url = {http://web.cs.du.edu/~sturtevant/papers/benchmarks.pdf},
}

Research papers using these sets If you have used any of these benchmarks in a published paper, send me the reference and I will include it here. (sturtevant *at* cs -dot- du -dot- edu)

Information on file formats

Benchmark sets

Commercial Game Benchmarks
BoxSourceBrowseMapBenchmarks
Dragon Age: Origins (published November, 2009) Browse data Download
all maps
Download
all benchmark problems
Warcraft III (published July, 2002)
Scaled to 512x512
Browse data Download
all maps
Download
all benchmark problems
Baldurs Gate II (published September, 2000)
Scaled to 512x512
Browse data Download
all maps
Download
all benchmark problems
Baldurs Gate II (published September, 2000)
Original maps
Browse data Download
all maps
Starcraft (published March, 1998) Browse data Download
all maps
Download
all benchmark problems

Artificial Benchmarks
ThumbnailSourceBrowseMapBenchmarks
Mazes; corridor width 1
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Mazes; corridor width 2
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Mazes; corridor width 4
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Mazes; corridor width 8
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Mazes; corridor width 16
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Mazes; corridor width 32
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Random obstacles; Map is exactly 10% filled
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Random obstacles; Map is exactly 15% filled
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Random obstacles; Map is exactly 20% filled
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Random obstacles; Map is exactly 25% filled
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Random obstacles; Map is exactly 30% filled
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Random obstacles; Map is exactly 35% filled
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Random obstacles; Map is exactly 40% filled
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Room maps; rooms size 8x8 with 80% of doors opened
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Room maps; rooms size 16x16 with 80% of doors opened
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Room maps; rooms size 32x32 with 80% of doors opened
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems
Room maps; rooms size 64x64 with 80% of doors opened
Nathan Sturtevant / HOG2
Browse data Download
all maps
Download
all benchmark problems

Most maps and benchmark sets are being maintained as part of the HOG2 repository at google code. They can be checked out using SVN as follows:

svn checkout http://hog2.googlecode.com/svn/trunk/maps map-read-only
svn checkout http://hog2.googlecode.com/svn/trunk/scenarios scenarios-read-only