Associate Director – Data Engineer – Tech-AI

ROLE OVERVIEW PURPOSE OF JOB As JTC enters a new phase of growth under private equity ownership, trusted and reusable data has become the foundation for the Group's ambitions. The Associate Director – Data Engineer is a senior, hands-on builder in the new Group Data Function, which delivers JTC's ONE DATA Strategy 2030. The postholder will design, build and run the data pipelines, models and data services on JTC's Databricks lakehouse. This includes taking data from core business systems such as Viewpoint, NavOne, QCore, Salesforce and others into a governed Common Data Foundation. That foundation supports reporting automation, self-serve insight and the Group's priority AI use cases. This is a senior, hands-on engineering role. The postholder will lead the technical delivery of work packages within the Data Platform Build, move legacy feeds and manual reconciliations onto the target platform, speed up the integration of acquired businesses, and treat internal data consumers as clients. They will work closely with Analytics Reporting and AI Engineering Data Science, including working iteratively with data scientists on ML and AI use cases. MAIN RESPONSIBILITIES AND DUTIES DATA PLATFORM COMMON DATA FOUNDATION Design, build, test and maintain scalable batch and incremental pipelines on Databricks, using PySpark, Spark SQL, Delta Lake and Lakeflow / Delta Live Tables. Put the medallion architecture into practice, and build reusable, well-documented data models for reporting, analytics and AI. Help build the Common Data Foundation: put standard data definitions and conformed entities (client, entity, fund, structure, account, transaction) in place across JTC businesses. Build reusable, governed data products that serve many consumers (AI, applications, automation, reporting and analytics) rather than one-off pipelines and extracts. Retire legacy data stores, manual reconciliations and point-to-point feeds by moving them onto the target platform. Apply software engineering practice to data work: Git-based version control, code review, automated testing, CI/CD, observability and Databricks Asset Bundles or Terraform. Design for scale, resilience and maintainability across JTC's multi-jurisdiction technology estate, following Group engineering standards and patterns. Shape the technical design of the platform: set engineering standards and reusable patterns and contribute data engineering input to architecture decisions. DATA GOVERNANCE, QUALITY SELF-SERVE ENABLEMENT Implement Unity Catalog controls: catalogue structure, access permissions, lineage, tagging and data classification. Build data quality checks, monitoring and alerts into pipelines, so data is trusted by design rather than fixed downstream. Track and fix issues with business data owners. Publish curated, documented datasets so that business users can self-serve trusted data. Make sure data handling meets JTC's regulatory, confidentiality and client obligations, including GDPR and the Jersey Data Protection Law. Work with Data Governance, Architecture, Cyber and Risk so that data products meet JTC's security, regulatory and control requirements. DATA PROVISIONING, INTEGRATION INTEROPERABILITY Build ingestion and integration from core business systems, including Viewpoint, NavOne, QCore and Salesforce, with other finance, HR and operational platforms coming into scope over time. Build and support API-based data services and data products, with clear service standards for internal consumers. Develop repeatable ingestion and mapping patterns for M A, so that acquired businesses can be integrated onto the JTC platform quickly. Build reusable tooling and automation for acquisition data discovery, migration, mapping, reconciliation and validation, with full traceability into Group reporting, analytics and AI. Provide trusted data for financial, commercial, operational and people reporting. This supports reporting automation, a faster month-end close and less duplicated transformation logic. AI-READY DATA Build governed pipelines for structured and unstructured data, such as documents, PDFs and correspondence, ready for analytics and AI use cases. Work with AI Engineering Data Science on grounding, retrieval, document intelligence, conversational analytics and model input quality. Work iteratively with data scientists to build and refine training, feature and evaluation datasets for ML and AI use cases. Provide secure, permissioned data access for JTC's agentic platform and AI assistants, with full lineage and auditability. Design data products for both human and machine consumption. Use reusable, secure access patterns for AI agents and digital products rather than a bespoke integration for each use case. Help move agentic capabilities from prototype into production on reliable, observable and scalable data foundations. Operations Ways of Working Monitor, troubleshoot and tune pipeline performance and Databricks compute cost. Write and maintain technical documentation, runbooks and data contracts. Work in an Agile team, turning business requirements into practical technical solutions. Use AI coding assistants to speed up development and testing and check every generated output carefully. Review designs and code, coach other engineers, and share good practice across the Group Data Function. E SSENTIAL REQUIREMENTS Proven commercial experience as a data engineer building and supporting production-grade ETL/ELT pipelines. Experience as a technical lead: setting engineering standards, reviewing designs and code, and mentoring other engineers, while staying hands-on. Strong hands-on Databricks experience, including Apache Spark / PySpark, Delta Lake, Databricks SQL, Workflows / Lakeflow Jobs and Unity Catalog. Excellent SQL and production-quality Python. Strong data modelling skills (conceptual, logical and physical; dimensional and medallion) and an understanding of lakehouse and warehouse architecture. Strong Azure experience, including ADLS Gen2, Entra ID, Key Vault and networking. Strong AWS or GCP engineers who can transfer their skills will also be considered. Experience with Git, CI/CD and DevOps / DataOps practice, such as Azure DevOps or GitHub Actions, automated testing and infrastructure-as-code. Experience integrating source systems through APIs, database extracts, file feeds and change data capture, including replacing legacy point-to-point flows. Experience building reusable data products and governed data-access patterns for multiple consumers, rather than one-off pipelines and extracts. Understanding of the data needs of AI and GenAI systems, including structured and unstructured data, retrieval and grounding. Practical experience of data quality, metadata and access controls in a regulated environment. Ability to explain technical concepts clearly to non-technical colleagues and to treat data consumers as clients. OUR COMMITMENT TO INCLUSION WELLBEING JTC is committed to fostering a healthy, inclusive organisation where all individuals feel welcome and feel able to participate in the workplace fully. We value different perspectives, backgrounds and lived experiences. This includes supporting employee wellbeing so that people feel equipped to thrive.