Staff Software Engineer - Distributed Data Systems

Tech Stack

Responsibilities

• Build the next generation distributed data storage and processing systems that can outperform specialized SQL query engines in relational query performance, yet provide the expressiveness and programming abstractions to support diverse workloads ranging from ETL to data science.
• Develop the de facto open source standard framework for big data with Apache Spark™.
• Deliver reliable and high performance services and client libraries for storing and accessing humongous amount of data on cloud storage backends, e.g., AWS S3, Azure Blob Store.
• Simplify the complexity of real-world data engineering architecture with Delta Lake by providing higher level abstractions and guarantees, including ACID transactions and time travel.
• Make it simple to orchestrate and operate tens of thousands of data pipelines with Delta Pipelines, enabling customers to deploy, test & upgrade pipelines and eliminate operational burdens.

Soft Skills

Benefits

• Equity
• Mental Health

Culture

Requirements