Senior Machine Learning Engineer - Model Evaluations, Public Sector

Tech Stack

Responsibilities

• Develop and maintain automated evaluation pipelines for ML models across functional, performance, robustness, and safety metrics, including LLM-judge–based evaluations.
• Design test datasets and benchmarks to measure generalization, bias, explainability, and failure modes.
• Build evaluation frameworks for LLM agents, including infrastructure for scenario-based and environment-based testing.
• Conduct comparative analyses of model architectures, training procedures, and evaluation outcomes.
• Implement tools for continuous monitoring, regression testing, and quality assurance for ML systems.

Soft Skills

Benefits

• 401k
• Commuter Benefits
• Dental
• Equity
• Flexible PTO
• Health Insurance
• Learning Budget
• Parental Leave
• Vision

Culture

Requirements