Senior Software Developer (Cloud Platform)
Posted Updated
Description
- Build and operate our data lake on a ClickHouse-centered architecture, including ingestion pipelines, data modeling, partitioning, and replication
- Develop and maintain data pipelines and event-driven services that move messages and data across the platform
- Implement data analytics jobs and APIs that expose data to product and application teams
- Partner with Application teams to understand requirements and recommend sound approaches to data design and access patterns
- Improve the performance, scalability, reliability, and cost efficiency of data services. This includes measurable gains in latency, throughput, and availability
- Implement data protection, security, and privacy controls for data processed from the edge to the core
- Build with modern, asynchronous Python and Golang, and use other languages where they fit
- Write and maintain documentation, including designs, evaluation plans, test results, and technical manuals
- Experience with cloud-native technologies: Kubernetes, containers, CI/CD, and observability
- Understanding of networking fundamentals (TCP/IP, DNS, HTTP/S) and distributed systems design
- Experience with data lake architectures and open file and table formats such as Parquet and Apache Iceberg (Delta Lake is a plus), on object storage
- Strong communication and teamwork skills
- 5+ years of software engineering experience, with 3+ years building data platforms or data-intensive distributed systems
- Passion for building solid REST APIs and distributed services
- Experience with data processing and analytics tools such as Pandas, Polars, DuckDB, or equivalents (Dask, Ray, Prefect are a plus)
- Experience building data pipelines on messaging and streaming systems (Kafka, RabbitMQ, Redis, or similar)
- Highly desirable: Kubernetes certification (CKAD or CKA)
- Strong production experience with Python (including async programming) and/or Golang
- Hands-on experience with ClickHouse or a comparable OLAP database or query engine (DuckDB, Druid, Pinot, Trino, or similar). This includes data modeling, ingestion, and query performance tuning
- Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent experience
- Experience with AWS (GCP and Azure are a plus)
- Working knowledge of the SDLC, Agile, testing strategies, and secure engineering practices
Skills
- Agile
- Analytics
- API
- AWS
- Azure
- CI/CD
- ClickHouse
- Cloud
- Cloud Native
- Dask
- Data Analytics
- Data Lake
- Data Modeling
- Data Pipelines
- Delta Lake
- Distributed Systems
- DNS
- Event Driven Architecture
- GCP
- Go
- Iceberg
- Kafka
- Kubernetes
- Networking
- Observability
- OLAP
- pandas
- Parquet
- Polars
- Prefect
- Python
- RabbitMQ
- Redis
- REST
- SDLC
- TCP/IP
- Trino