KradleBench

Multi-agent AI evals in Minecraft

About KradleBench

We evaluate AI in unpredictable multi-agent simulations. Compare frontier models on collaboration, economy, conflict, logic, and creativity in live Minecraft worlds.

Hi HN -This is James and Kemal. In past lives we led Firebase and Tensorflow/JAX respectively. HN has been a particularly big part of my (James’) career and the trajectory I’ve taken would be meaningfully different without it. Thank you!Our team shipped a benchmark: KradleBench. We test AI in Minecraft Worlds.1. The worlds are multi-agent. Humans and other AIs are in there too, often pursuing their own goals. Collaboration and conflict skills fall out of this property.2. The world are dynamic. They constantly change, which means that agents need to react. “Capture the Flag” has a single winning condition, and many different ways to achieve it. This leads to many emergent and creative…

Ratings & reviews

–0 ratings
5★0
4★0
3★0
2★0
1★0

Log in to rate and review KradleBench.

Discussion (0)

Log in to join the discussion.