Member of Technical Staff, Evals

Tech Stack

Responsibilities

• Design and build evaluation frameworks, benchmarks, and methodologies for frontier LLMs, AI agents, multimodal models, and reinforcement-learning environments.
• Develop reward models, programmatic verifiers, graders, and other feedback systems that make model behavior measurable and improvable.
• Research what makes evaluations representative, difficult, reliable, and resistant to shortcutting or reward hacking.
• Build systems for high-quality human data, including expert task design, annotation methodologies, data-quality signals, and data-attribution techniques.
• Run fast, rigorous iteration loops: prototype, evaluate, interpret results, diagnose failure modes, and turn learnings into the next benchmark or system.

Soft Skills

Benefits

• 401k
• Commuter Benefits
• Dental
• Equity
• Flexible PTO
• Gym/Wellness
• Health Insurance
• Learning Budget
• Mental Health
• Parental Leave
• Vision

Culture

Requirements