Senior Data Engineer
Posted Updated
Description
- We’re looking for a Senior Data Engineer with deep expertise in healthcare data to join a pioneering team building data products that enable world-leading research.
- Whether your experience comes from academia, healthcare, biotech, or a fast-growing health-tech startup, you’ll bring a strong understanding of healthcare data standards such as OMOP or FHIR, alongside the technical leadership required to be a Senior Engineer
- In this role, you’ll design and build scalable data pipelines that transform complex health records into trusted, research-ready datasets used by scientists and researchers tackling challenges with global significance.
- You’ll work at the intersection of engineering, data science and epidemiology, helping to establish best practices, shape technical direction and create reusable, high-quality data products that can operate at industrial scale
- Designing and building scalable data pipelines that ingest healthcare data from multiple providers into trusted research environments and data platforms
- Developing and maintaining data transformation logic in Python to create high-quality, curated and research-ready datasets
- Contributing to complex, shared codebases while following engineering best practices including code reviews, testing and clean coding standards
- Collaborating with data scientists, epidemiologists, software engineers and product teams to understand requirements and deliver impactful data solutions
- Processing large-scale healthcare datasets and implementing robust data quality, validation and governance controls
- Transforming and harmonising healthcare data into common data models such as OMOP and FHIR to support research and analytical use cases
- Creating reusable, scalable and maintainable data products that enable efficient consumption by multiple stakeholders
- Communicating effectively with technical and non-technical stakeholders to translate complex requirements into practical engineering solutions
- Producing clear and comprehensive technical documentation covering data pipelines, transformation logic, architecture and operational processes
- Supporting the design, scoping and implementation of data flows across the wider healthcare data ecosystem
- Leveraging cloud and big data technologies, including Azure, Databricks, Spark and Data Lakes, to deliver high-performing data solutions
- Adopting and promoting data engineering best practices, healthcare data standards and continuous improvement across the team
- Keeping abreast of emerging trends, technologies and standards within healthcare, research and data engineering to drive innovation and excellence
Benefits
- 30 days annual leave plus bank holidays.
- Generous company pension package with employer contributions of up to 12%
- We have a lovely new office in Holborn, Central London and we offer flexible and remote working arrangements.
- We offer all employees a life assurance policy.
- Training support to help you achieve your potential.
- All employees are covered by a group income protection scheme to support them in the event of long term sickness.
- Experience designing, building and maintaining scalable, reliable data pipelines that process large volumes of data from multiple sources
- Knowledge of healthcare data standards and common data models, including OMOP and/or FHIR
- Experience working with cloud platforms, ideally Microsoft Azure
- Good command-line and Unix/Linux skills
- Knowledge of data storage and transformation formats such as Apache Parquet and Delta Lake
- Experience working within Agile delivery teams
- Understanding of healthcare terminologies and ontologies, including SNOMED CT, ICD-10 and Read Codes
- Experience transforming and remodelling data between different Common Data Models (CDMs)
- Experience using Git/GitHub and working within collaborative development environments
- Experience working with healthcare data across academia, healthcare, life sciences, health technology or related sectors
- Experience with containerisation and orchestration technologies, including Docker and Kubernetes
- Experience contributing to shared codebases and developing reusable, maintainable data solutions
- Experience using technologies such as Spark, Databricks and data lake architectures
- Proficiency in Python and software engineering best practices
- Proven Data Engineering experience within complex, large-scale data environments
