Machine Learning Data Systems

Tech Stack

Responsibilities

• Design and ship replacements for core data pipelines that cannot guarantee necessary properties, while keeping existing systems running.
• Define and implement instrumentation for new product surfaces to ensure all necessary data is captured.
• Identify and fix instrumentation gaps that lead to drops in eval coverage, add contracts to prevent recurrence, and implement dashboards for early detection.
• Design schema evolution and validation for overlapping data dependencies to prevent silent degradation across consumers.
• Manage storage costs by deciding what data to retain, implementing retention and compression, and deleting unnecessary data.

Culture

Requirements