Staff Software Engineer, Foundational Model Serving

Tech Stack

Responsibilities

• Design and implement core systems and APIs that power Databricks Foundation Model Serving, ensuring scalability, reliability, and operational excellence.
• Partner with product and engineering leadership to define the technical roadmap and long-term architecture for serving workloads.
• Drive architectural decisions and trade-offs to optimize performance, throughput, autoscaling, and operational efficiency for GPU serving workloads.
• Contribute directly to key components across the serving infrastructure — from working in systems like vLLM and SGLang to creating token based rate limiters and optimizers — ensuring smooth and efficient operations at scale.
• Establish best practices for code quality, testing, and operational readiness, and mentor other engineers through design reviews and technical guidance.

Benefits

• Equity
• Health Insurance

Culture

Requirements