RedlineBench (Crosby)
redlinebench · v1 · harbor · judge Claude Haiku 4.5 (judge)
Contract-negotiation redlining: 140 tasks across 3 MSA negotiations and 4 turns, where the agent, as in-house counsel, produces a Word redline with tracked changes and comments. Graded against attorney-authored weighted rubrics by a 3-judge LLM panel (GPT-5.4 mini, Claude Haiku 4.5, Gemini 3.1 Flash-Lite; majority vote). Official Harbor tasks on Modal; Claude Code for Anthropic models, Codex for OpenAI models, Terminus-2 otherwise.
New run
Sign in to see estimates and start runs.