Skip to content

People Data Labs

NewBe an early applicant

Senior Software Engineer (Data Acquisition)

Posted Updated

Description

  • We are looking for individuals who can balance extreme ownership with a “one-team, one-dream” mindset
  • Our customers are trying to solve complex problems, and we only help them achieve their goals as a team
  • Our Data Engineering & Acquisition Team ensures our customers have standardized and high quality data to build upon
  • You will be crucial in accelerating our efforts to build standalone data products that enable data teams and independent developers to create innovative solutions at massive scale
  • In this role, you will be working with a team to continuously improve our existing datasets as well as pursuing new ones
  • If you are looking to be part of a team discovering the next frontier of data-as-a-service (DaaS) with a high level of autonomy and opportunity for direct contributions, this might be the role for you
  • We like our engineers to be thoughtful, quirky, and willing to fearlessly try new things
  • Failure is embraced at PDL as long as we continue to learn and grow from it
  • Contribute to the architecture and improvement of our data acquisition and processing platform, increasing reliability, throughput, and observability
  • Use and develop web crawling technologies to capture and catalog data on the internet
  • Build, operate, and evolve large-scale distributed systems that collect, process, and deliver data from across the web
  • Design and develop backend services that manage distributed job orchestration, data pipelines, and large-scale asynchronous workloads
  • Structure and model captured data, ensuring high quality and consistency across datasets
  • Continuously improve the speed, scalability, and fault-tolerance of our ingestion systems
  • Partner with data product and engineering teams to design and implement new data products powered by the data you help collect, and enhance and improve upon existing products
  • Learn and apply domain-specific knowledge in web crawling and data acquisition, with mentorship from experienced teammates and access to existing systems

Benefits

  • Remote-first work, always: As a remote-first company, we believe in flexible work that fits your life. It’s that simple.
  • Unlimited PTO: No checking how many vacation days you have left! Rest, recharge, and take time off when you need it.
  • Monthly Employee Stipends: We provide monthly stipends automatically for employees. Think internet bills bills, running shoes, your new pickleball obsession, etc. — use it as you wish!
  • Medical, Dental, Vision Insurance: We provide multiple insurance plan options to make sure you and your family are covered!
  • Experience with streaming data systems (Kafka, Pub/Sub, Spark Streaming, etc.)
  • Scope and break down complex projects into deliverable milestones, and communicate progress, risks, and blockers effectively
  • Proficiency with SQL and data warehousing (Snowflake, Redshift, BigQuery, or similar)
  • Experience as a Red Teamer
  • Balance pragmatism with craftsmanship, shipping reliable systems while continuously improving them
  • Communicate clearly and thoughtfully in writing (Slack, docs, design proposals)
  • Degree in a quantitative field such as computer science, mathematics, or engineering
  • Familiarity with network architecture and debugging (HTTP, DNS, proxies, packet capture and analysis)
  • Solid understanding of distributed systems concepts: parallelism, asynchronous programming, backpressure, and message-driven design
  • Familiarity with message queues, orchestration, and distributed task systems (Kafka, SQS, Airflow, etc.)
  • Experience working on large-scale data ingestion, crawling, or indexing systems
  • Experience evaluating and monitoring data quality, ensuring consistency, completeness, and reliability across releases
  • Strong grasp of software architecture and backend fundamentals; you can reason clearly about concurrency, scalability, and fault tolerance
  • Proficiency with Linux / Unix command-line tools and system resource management
  • Experience designing or maintaining resilient data ingestion, API integration, or ETL systems
  • Solid programming experience in Python, Go, Rust, or similar, including experience with async / await, coroutines, or concurrency frameworks
  • Work independently in a fast-paced, remote-first environment, proactively unblocking themselves and collaborating asynchronously
  • Experience with cloud platforms (AWS preferred, GCP or Azure also great)
  • Experience with Apache Spark, Databricks, or other distributed data platforms
  • Solid understanding of browser rendering pipeline, web application architecture (auth, cookies, http request / response)
  • 7+ years of professional experience building or operating backend or infrastructure systems at scale
  • Write and maintain technical design documents, including pipeline design, schema design, and data flow diagrams
  • Knowledge of modern data design and storage patterns (e.g., incremental updating, partitioning and segmentation, rebuilds and backfills)
  • Experience building and maintaining data pipelines on modern big-data or cloud platforms (Databricks, Spark, or equivalent)
  • Understanding of modern data storage and design patterns (parquet, Delta Lake, partitioning, incremental updates)

Skills

What Senior Software Engineering jobs ask for — and how much of it you have →
Apply

Apply to Senior Software Engineer (Data Acquisition) at People Data Labs

Create a free account first, or go straight to the employer.

Free RoleWatch account

  • Get an email when new jobs match your search
  • Save jobs and track every application
  • See how well your CV matches each job

Already have one?

No account needed. Opens the employer's own site in a new tab.

See also

Software Engineering jobs by country — openings, pay and top skills →