ProgramBench
programbench · v1 · repo
200 tasks: rebuild a program from scratch given only its compiled binary and docs, graded by hidden behavioural tests. Official harness (mini-swe-agent + programbench eval) inside a Docker-capable Modal VM per task; mini-swe-agent for every model, as on the leaderboard. Headline: % resolved (all tests pass).
New run
Sign in to see estimates and start runs.