Senior Software Engineer in Test (AI Agentic Systems)

Tech Stack

Responsibilities

• Build and maintain a versioned library of "Grounding Data" results to define "Ground Truth."
• Design automated "LLM-grading-LLM" workflows using custom rubrics to score factual grounding and policy compliance.
• Develop testing libraries to validate semantic equivalence and numerical accuracy in agent outputs.
• Programmatically verify that mandatory tools were invoked with correct arguments using Vertex AI traces.
• Integrate automated regression runs with CI/CD for every prompt or model update in Vertex AI Prompt Management.

Soft Skills

Benefits

• Health Insurance
• 401k
• Flexible PTO
• Equity
• Hybrid Work
• Internal Mobility
• Mentorship Program
• Learning Budget

Culture

Requirements