Software Engineer - GenAI inference

Tech Stack

Responsibilities

• Contribute to the design and implementation of the inference engine for large-scale LLM inference.
• Collaborate with researchers to integrate new model architectures and features into the engine.
• Optimize for latency, throughput, memory efficiency, and hardware utilization across GPUs and accelerators.
• Build and maintain instrumentation, profiling, and tracing tooling to identify and resolve performance bottlenecks.
• Develop and enhance scalable routing, batching, scheduling, memory management, and dynamic loading for inference workloads.

Benefits

• Equity

Culture

Requirements