Data Engineer
Posted Updated
Description
- The Data Engineering Team is central to the company’s work. This role offers autonomy and the opportunity to contribute directly to the next frontier of data-as-a-service
- Build infrastructure to ingest, transform, and load an exponentially increasing volume of data from a variety of sources using Spark, SQL, AWS, and Databricks
- Build an organic entity resolution framework capable of correctly merging hundreds of billions of individual entities into clean, consumable datasets
- Develop CI/CD pipelines and anomaly detection systems to continuously improve the quality of data pushed into production
- Develop solutions to undefined data engineering and data science problems
Benefits
- Remote-first work, always: As a remote-first company, we believe in flexible work that fits your life. It’s that simple.
- Unlimited PTO: No checking how many vacation days you have left! Rest, recharge, and take time off when you need it.
- Monthly Employee Stipends: We provide monthly stipends automatically for employees. Think internet bills bills, running shoes, your new pickleball obsession, etc. — use it as you wish!
- Medical, Dental, Vision Insurance: We provide multiple insurance plan options to make sure you and your family are covered!
- Experience working with entity data (entity resolution or record linkage)
- Ability to balance high ownership and autonomy with strong collaboration
- Strong software development fundamentals
- Understanding of modern data storage formats and tools, such as Parquet, ORC, Avro, and Delta Lake
- Ability to scope and break down projects and communicate progress and blockers effectively with managers, teammates, and stakeholders
- Degree in a quantitative discipline such as computer science, mathematics, statistics, or engineering
- Ability to work effectively remotely, proactively managing blockers, asking questions, and participating in team activities
- 3–5+ years of industry experience, with clear examples of strategic technical problem-solving and implementation
- Experience building scalable data processing systems (e.g., cleaning and transformation) from the ground up
- Experience writing data design documents, including pipeline design, dataflow, and schema design
- Expertise with Apache Spark (Java, Scala, and/or Python-based)
- Expertise with Python and the Python data stack, such as NumPy and pandas
- Experience using developer-oriented data pipeline and workflow orchestration tools (e.g., Airflow, dbt, Dagster, or similar)
- Experience working with data acquisition or data integration
- Experience with streaming platforms such as Kafka
- Strong written communication skills in Slack/chat and documents
- Experience with SQL
- Experience with data warehousing, such as Databricks, Snowflake, Redshift, or BigQuery
- Experience evaluating data quality and maintaining high data standards across new feature releases, including consistency, accuracy, validity, and completeness
- Knowledge of modern data design and storage patterns, such as incremental updating, partitioning and segmentation, rebuilds, and backfills
- Experience working in Databricks, including Delta Live Tables and data lakehouse patterns
- Experience with Python
- Experience with cloud computing services such as AWS, GCP, or Azure
Skills
Sign in to read and join the discussion.