Inference Technical Lead, On-Device Transformers

Tech Stack

Responsibilities

• Evaluate and select silicon platforms (GPUs, NPUs, and specialized accelerators) for on-device and edge deployment of OpenAI models.
• Work closely with research teams to co-design model architectures that meet real-world deployment constraints such as latency, memory, power, and bandwidth.
• Analyze and model system performance, identifying tradeoffs between model design, memory hierarchy, compute throughput, and hardware capabilities.
• Partner with hardware vendors and internal infrastructure teams to bring up new accelerators and ensure efficient execution of transformer workloads.
• Build and lead a team of engineers responsible for implementing the low-level inference stack, including kernel development and runtime systems.

Soft Skills

Benefits

• Relocation Assistance

Culture

Requirements