Senior Data Engineer
Posted Updated
Description
- We are looking for a Senior Data Engineer to join our Data Automation team
- You will play a critical role in designing and building scalable data platforms and pipelines that power Socure’s identity verification products and analytics
- Design and build batch and streaming data pipelines to support automated data ingestion, ML feature engineering and analytics across multiple product domains
- Own end-to-end delivery of complex, ambiguous data initiatives, including architecture, implementation, testing, deployment, monitoring, and documentation
- Develop and evolve the data platform to support large-scale data processing using modern cloud-native technologies
- Automate data operations (validation, quality checks, alerting, backfills, and recovery workflows) to reduce manual effort and improve consistency
- Optimize cost, performance, and reliability of data workloads
- Partner closely with cross-functional teams (Data Science, Product, Engineering) to understand requirements, translate them into technical solutions
- Evaluate and adopt new technologies (new processing engines, storage formats, orchestration tools, GenAI-assisted ingestion) to keep the platform modern and efficient
Benefits
- Health: Comprehensive medical coverage, including vision, dental, and FSA
- Parental Leave: Generous leave allowance for
parents-to-be - 401k: Pre-tax savings for retirement
- Life Insurance: Protect your family in
case of emergency - PTO: Take as much time as you need,
when you need it - 100% Remote: Work from anywhere anytime, whatever suits your lifestyle
- WFH Allowance: Generous reimbursement program to maximize your WFH setup
- Deskpass: Work in an office environment whenever you feel the need
- Mental Health Provider: Connect with clinicians when you need it
- Learning & Development: Self-education allowance & free career coaching
- Employee Assistance Program “EAP”: Lifestyle resources available as well as additional mental health resources
- Fertility Provider: Family planning solutions for those interested
- Personal Legal Support: Significant discounts with RocketLawyer
- This role is ideal for someone who has a strong passion for solving real business problems with data, and combines deep hands-on data engineering expertise with strong ownership
- Experience operating and supporting production pipelines, including monitoring, alerting, incident response, and improving reliability over time
- Solid foundation in software engineering practices, including version control, CI/CD, testing strategies, and code review
- Deep experience with distributed data processing frameworks, such as Apache Spark, including performance tuning and optimization
- Strong understanding of data modeling and data warehousing concepts, including partitioning, schema design for large-scale datasets
- 5+ years of hands-on data engineering experience, building and maintaining production-grade data platforms and pipelines
- Strong programming skills in general-purpose language (such as Python or Scala) for data processing, and SQL for data analytics
- Proven experience building data solutions using services on AWS (EMR, Lambda, s3, etc)
- Experience with streaming or near-real-time data processing (Kafka, Kinesis, etc)
- Hands-on experience with data orchestration tools (Airflow, Step Functions, etc)
- Familiarity with modern data platform patterns such as Data Lakehouse, Data Mesh, and large-scale data sharing across teams
- Experience with prompt engineering using modern GenAI, Large Language Models (LLM)
- Experience mentoring other engineers and contributing to engineering-wide standards, best practices
- Strong communication and collaboration skills, with the ability to work effectively with both technical and non-technical stakeholders
Skills
- Airflow
- Analytics
- Automation
- AWS
- CI/CD
- Cloud
- Cloud Native
- Data Analytics
- Data Engineering
- Data Ingestion
- Data Mesh
- Data Modeling
- Data Pipelines
- Data Science
- Data Warehousing
- Feature Engineering
- Generative AI
- Kafka
- Kinesis
- Lakehouse
- Lambda
- LLM
- Machine Learning
- Prompt Engineering
- Python
- S3
- Scala
- Spark
- SQL
- Version Control
Sign in to read and join the discussion.