Research Scientist, Agent Evaluation

The role

Knowing whether an agent improved is harder than it sounds. You’ll design evaluations of agents on long-horizon, tool-using tasks that are statistically sound, contamination-resistant and diagnostic about why a model failed.

What you’ll own

Held-out evaluation suites across our environment library.Metrics beyond pass rate: efficiency, recovery from errors, calibration, safety of actions.Contamination controls and leakage detection between training and evaluation.Failure-mode analysis and the reports customers act on.Published benchmarks and methodology.

What you bring

A research background in evaluation, benchmarking or agents.Strong statistics and experimental design.Fluent Python and experience with evaluation harnesses.Clear technical writing.

Helpful, not required

A public benchmark you released.Domain depth in coding, finance or science.