Senior Software Engineer (Data Ingestion)
Posted Updated
Description
- Design and develop highly scalable and resilient streaming and batch pipelines for ingesting and processing structured and unstructured datasets
- Design and implement Slack’s Lakehouse solutions and enable production use cases for customers
- Improve the reliability and performance of the services/pipelines through AI assisted and/or Agentic solutions and tools
- Automate and handle the lifecycle of datasets (schema evolution, metadata management, change and backfill management, deprecation and migration)
- As one of the tech leads, comfortably collaborate with cross functional partners and lead technical initiatives independently end to end
- Be a role model and a multiplier, coaching and mentoring other engineers across the team and the org
- Write, review, or provide feedback on a technical design proposal from others
Benefits
- Medical Care
- Life Insurance
- Retirement Savings
- Employee Assistance Programs
- With 9 standard holidays and four floating holidays, you get a total 13 paid days off each year
- 7+ years of software/data engineering experience, including experience with building applications using Streaming and Lakehouse technologies, e.g. Kafka/Kafka Connect, Flink/Spark Streaming, Iceberg/Hudi/Delta or equivalent
- You are proficient in object-oriented and/or functional programming languages: SQL, Python, Java/Scala, Go or equivalent
- You are skilled at crafting and building robust distributed microservices with tools like Kubernetes, Docker, AWS ECS/EKS, Terraform, Grafana, etc
- U.S. citizenship and willing to undergo a background check for GovSlack authorization. We can not provide visa sponsorship for this role
- You have hands on experience and knowledge on building and maintaining batch data pipelines using Spark, Airflow, EMR, S3 or equivalent
- You are familiar with AI-assisted software development and automation and have hands on experience of Claude Code/Codex or equivalent
- You have excellent written and verbal communication and interpersonal skills; able to effectively collaborate with cross functional partners and explaining sophisticated technical concepts to non-technical stakeholders
- You have high growth expectations for yourself and your team, and a willingness to push yourself and your team to achieve them
Skills
Sign in to read and join the discussion.