Member of Technical Staff - Pre-Training Evals Infra

Tech Stack

Responsibilities

• Design and evolve clear evaluation abstractions, APIs, and data models for evaluation tasks and configurations
• Build scalable, reliable distributed systems and execution infrastructure for model evaluation throughout training and release
• Optimize evaluation and inference performance, profiling servers to improve GPU utilization and throughput
• Establish evaluation correctness, reproducibility, versioning, and regression testing for datasets and outputs
• Develop researcher-facing tools, intuitive interfaces, and dashboards for inspecting model outputs and debugging

Benefits

• Equity
• Gym Membership
• Health Insurance
• Parental Leave

Culture

Requirements