Skip to content

robo-evals

Seeded, reproducible success rates for robot manipulation policies in MuJoCo, with confidence intervals and episode videos.

A terminal session: version, task list, a random run, a scripted run with video, and a comparison

The recording above is a real session with the released command line, not a mock-up.

Install

pip install "robo-evals[video]"

Run it

robo-evals run --policy scripted --suite core --episodes 20 --video first
robo-evals compare results/random/report.json results/scripted/report.json

Each run writes report.json, report.md, and videos under results/<policy>/.

Where to go next

  • Architecture shows how a run flows from the command line to a report.
  • Remote policy protocol explains how to evaluate a model in another process.
  • FAQ answers the common questions about seeds, intervals, and rendering.
  • Decisions records why the harness works the way it does.