Model Evaluation QA Lead

Tech Stack

Responsibilities

• Design, build, and maintain automated model evaluation pipelines for candidate models, implementing objective and subjective quality metrics across STT, TTS, and STS products.
• Embed model quality checkpoints into CI/CD and release pipelines, defining pass/fail criteria and owning the go/no-go signal for production promotions.
• Stand up and operate evaluation tooling for end-to-end voice agent testing, covering accuracy, latency, turn-taking, conversational quality, and custom metrics.
• Partner with the Active Learning team to validate data ingestion infrastructure, annotation pipelines, and retraining automation.
• Automate execution and reporting of industry-standard benchmarks and maintain reproducible benchmark environments across multiple model versions.

Benefits

• 401k
• Flexible Hours
• Gym Membership
• Health Insurance
• Learning Budget
• Parental Leave
• Remote Stipend
• Unlimited PTO

Culture

Requirements