Staff Software Engineer - GenAI inference

Tech Stack

Responsibilities

• Own and drive the architecture, design, and implementation of the inference engine for large-scale LLMs inference.
• Partner closely with researchers to integrate new model architectures or features into the engine.
• Lead end-to-end optimization for latency, throughput, memory efficiency, and hardware utilization across GPUs and accelerators.
• Define and guide standards for building and maintaining instrumentation, profiling, and tracing tooling.
• Architect scalable routing, batching, scheduling, memory management, and dynamic loading mechanisms for inference workloads.

Soft Skills

Benefits

• 401k
• Dental
• Disability Insurance
• Equity
• Health Insurance
• Life Insurance
• Mental Health
• Vision

Culture

Requirements