Staff Applied Scientist - Agentic Interfaces

Tech Stack

Responsibilities

• Own the evaluation strategy for Datadog's AI agent integrations, defining metrics for quality and cost across single-turn and trajectory-level interactions.
• Build eval datasets, golden traces, and regression harnesses, making these assets reusable for all teams contributing tools.
• Drive measurable improvements in retrieval relevance, tool-selection accuracy, and context efficiency, collaborating with AI engineers.
• Conduct applied research on agent-data interaction problems like tool selection in large catalogs, multi-turn agent evaluation, grounding, hallucination control, and cost/quality tradeoffs.
• Provide technical leadership within the Agentic Interfaces team and the broader organization through design reviews, mentorship, and external representation.

Benefits

• 401k
• Equity
• Gym Membership
• Health Insurance
• Learning Budget

Culture

Requirements