Data Scientist, Inference Capacity Optimization

Tech Stack

Responsibilities

• Build statistical and machine learning models to profile and improve GPU utilization, latency, throughput, and overall fleet efficiency.
• Develop forecasting models for inference demand across products, regions, and model families.
• Analyze production workloads to identify latency bottlenecks and capacity constraints.
• Partner with Capacity Systems Engineering to inform infrastructure planning and long-term GPU investment strategies.
• Design experiments and simulations to evaluate scheduling policies, serving strategies, and infrastructure tradeoffs.

Culture

Requirements