Tech Stack
Responsibilities
- Design AI evaluations by defining ground truth, metrics, and scoring methods for models and agents.
- Build repeatable evaluation loops to track quality over time and catch regressions before release.
- Partner with engineering and analytics engineering teams to operationalize evaluation harnesses.
- Translate evaluation results into actionable recommendations for system improvements.
- Establish quality standards, evaluation guidelines, documentation, and review practices.
Benefits
- Equity
Culture
Cross-Functional TeamsOpen Source ContributorsInclusive HiringHome Office Budget
Requirements
Preferred: A quantitative degree is a plus
Regions: Br, Ca, Us
Get jobs like this in your inbox
Weekly Python, Rust, TypeScript hiring trends and salary data — free.
Join 8 engineers getting weekly insights
Get job market intelligence in your inbox
Free weekly insights on tech hiring trends, salaries, and in-demand stacks.
Already a subscriber? Sign in
About Alpaca
Industry: fintech
Size: medium
Alpaca is a global leader in agent-first brokerage infrastructure, offering licensed financial services and institutional-grade APIs for stock, crypto, and multi-asset trading.
View company profile →Compensation
Equity: Stock Options
Similar Jobs