Skip to content

Octus

Principal Data Engineer

Posted Updated

Description

  • Octus is seeking a Principal Data Engineer to design, develop, and lead scalable data ingestion and transformation pipelines.
  • This is a hands-on Individual Contributor (IC) role that encompasses some management duties, including mentoring, architecture guidance, and providing technical leadership.
  • You’ll play a key role in designing and maintaining robust data infrastructure that powers data platforms, products and automation initiatives across the firm.
  • Lead the overarching technical strategy for the data platform, ensuring alignment between data infrastructure and long-term business goals
  • Lead the design and development of data ingestion and transformation pipelines, ensuring scalability, efficiency, and reliability across diverse data sources (APIs, web data, internal feeds, etc.)
  • Serve as a foundational technical leader and mentor for senior engineers within the data platform team, guiding architecture, design, and implementation decisions
  • Architect and manage data pipelines and orchestration workflows using AWS services such as MWAA (Airflow), Lambda, ECS, and SQS
  • Implement and maintain infrastructure as code (IaC) using Terraform, ensuring reproducibility and compliance with cloud standards
  • Partner with data analysts, scientists, and backend engineers to ensure data consistency, discoverability, and reliability
  • Apply best practices in data modeling, schema design, and ETL/ELT processes for high-volume structured and semi-structured data
  • Ensure data quality and lineage through automated testing, monitoring, and alerting
  • Promote continuous improvement through code reviews, observability practices, and team-wide knowledge sharing
  • Collaborate closely with technology leadership to align data platform development with business strategy and product goals
  • Stay up to date with industry trends in data engineering, cloud architecture, AI/ML integration, and automation

Benefits

  • Medical, dental, vision and 401(k) benefits
  • A flexible, hybrid working policy with home office reimbursements
  • Reimbursements for career development and continuing education
  • Generous paid time off – including vacations, sick leave and parental leave
  • Employee referral bonuses
  • Gym subsidies and other fun fitness benefits
  • Kudos program to recognize and reward colleagues
  • Manager training programs
  • Familiarity with CI/CD pipelines, automated testing, and modern DevOps practices
  • Expert proficiency in Python for data pipeline and automation development
  • Familiarity with containerization (Docker) and workflow orchestration best practices
  • Strong understanding of data ingestion, transformation, and orchestration tools and patterns, including those used in AI/ML pipelines
  • Extensive experience in a technical leadership capacity, including mentoring and leading complex data infrastructure projects end-to-end
  • Strong foundation in software engineering principles, including SOLID design, modularity, and scalability
  • Excellent communication, collaboration, and problem-solving skills
  • Experience with data warehousing or lakehouse technologies (Redshift, Snowflake, Databricks, etc.)
  • Advanced SQL skills and experience optimizing complex queries and data models
  • Experience implementing and managing Terraform or similar IaC frameworks
  • 8+ years of experience in data engineering or backend development, with a focus on scalable data solutions
  • Proven experience designing and maintaining cloud-native data pipelines on AWS (e.g., MWAA/Airflow, Lambda, ECS, SQS, Glue, S3, Redshift, etc.)
  • Exposure to streaming data technologies (Kafka, Kinesis, Flink)
  • Experience integrating data quality and observability tools (Great Expectations, Monte Carlo, etc.)
  • Familiarity with Scrapy, BeautifulSoup, or other data extraction frameworks for ingestion pipelines

Skills

What Principal Data Engineering jobs ask for — and how much of it you have →
Apply

Apply to Principal Data Engineer at Octus

Create a free account first, or go straight to the employer.

Free RoleWatch account

  • Get an email when new jobs match your search
  • Save jobs and track every application
  • See how well your CV matches each job

Already have one?

No account needed. Opens the employer's own site in a new tab.

See also

Data Engineering jobs by country — openings, pay and top skills →