Staff Software Engineer (Big Data Storage)
Posted Updated
Description
- We’re looking for a Staff Software Engineer to help build the next generation of Pinterest’s big data storage platform
- You’ll work on some of the most exciting big data open source technologies — especially Apache Iceberg — at exabyte scale to power the data infrastructure that helps Pinners discover and do what they love
- As a Staff Software Engineer, you’ll serve as a technical leader and hands-on contributor, designing and building highly scalable storage systems for Pinterest’s data lake
- You’ll partner closely with teams across data, ML/AI, analytics, and infrastructure to evolve our storage and metadata management capabilities, enabling efficient, reliable, and governed access to data at massive scale
- Design, implement, and optimize Pinterest’s exabyte-scale data lake storage platform
- Lead complex technical projects and initiatives for data lake storage and metadata management, driving them from architecture through execution
- Collaborate with stakeholders and partner teams across the organization to design storage and metadata layer technologies that unlock big data and ML/AI innovations
- Build storage capabilities that efficiently support large-scale ML/AI workloads, including high-throughput data access, schema evolution, and large-scale column backfills
- Shape the long-term technical direction for scalable, reliable, and efficient big data storage systems
- Engage with and contribute to open source communities such as Apache Iceberg, Spark, and Flink to help address Pinterest’s scaling challenges
- 8+ years of relevant industry experience designing and building large-scale production distributed systems
- Experience with big data technologies such as Apache Iceberg, Spark, Flink, Presto/Trino, Hive, or similar systems
- Proven ability to lead complex technical initiatives and influence architecture across teams
- Strong experience designing and maintaining scalable storage, metadata, or data lake infrastructure
- Experience building storage capabilities for large-scale ML/AI or analytics workloads, including high-throughput data access, schema evolution, and large-scale column backfills
- Proficiency in programming languages like Java, Scala, or Python
- Strong collaboration, communication, and problem-solving skills, with a drive for technical excellence and innovation
- Deep knowledge with building distributed systems, data storage systems, and production infrastructure
- Bachelor’s degree in a relevant field such as Computer Science, or equivalent experience