Lead DevOps/MLOps Engineer
Description
We are looking for a Lead DevOps/MLOps Engineer to join our Data Science Department and work with the ML Engineering team to build out and harden our on-premises, GPU-backed ML platform. In this role, you will help enable multiple ML and LLM use cases to run simultaneously, reliably, and with low latency, while scaling the platform to support a wider user base across the organization.
Responsibilities
- Build and harden the on-premises GPU-backed ML platform to support concurrent ML and LLM workloads
- Troubleshoot and resolve daily resource bottlenecks and latency issues across competing production and development use cases
- Deploy and maintain the NVIDIA GPU Operator and manage GPU resources across Kubernetes and OpenShift clusters
- Implement GPU sharing and isolation strategies to optimize memory and fault isolation across workloads
- Scale the GPU environment to accommodate growing compute demands as new H200 servers are added
- Maintain and improve platform services such as MLflow and the Agent Gateway
- Manage monitoring pipelines using NVIDIA DCGM exporter, Prometheus and Grafana to track GPU utilization and performance
- Collaborate with the Deep Learning and Traditional ML teams to support shared infrastructure, including Ray-based training workloads
- Ensure platform reliability and security given the confidential and sensitive nature of data handled on-premises
- Partner with the Data Analytics team to support the in-house agentic platform as it scales to a broader employee base
Requirements
- 5+ years of experience running GPU workloads on Kubernetes or OpenShift in production, ideally on-premises
- Practical experience deploying and troubleshooting the NVIDIA GPU Operator, with clear understanding of each component's function
- Understanding of how GPUs differ from CPUs as a scheduled resource, including memory limits, utilization semantics and scheduling limitations
- Knowledge of GPU sharing and isolation methods such as MIG, time-slicing and MPS, along with their trade-offs
- Solid Linux fundamentals, including cgroups, namespaces, systemd resource controls and process-level resource management
- Experience with Prometheus, Grafana and NVIDIA DCGM, including GPU metrics at process or pod level
- Working knowledge of the CUDA stack, including drivers, CUDA toolkit and container runtime version compatibility
- Experience deploying with Helm and operating MLOps tools such as MLflow and Airflow on Kubernetes
Nice to have
- Familiarity with LLM serving engines such as vLLM, SGLang or NVIDIA Triton
- Experience with Ray, KubeRay and GPU-aware schedulers such as Kueue or Volcano
- Hands-on work with H100/H200-class hardware and GitLab CI
Benefits
We connect like-minded people:
- Delivering innovative solutions to industry leaders, making a global impact
- Opportunity to work abroad for up to two months per year
We invest in your growth:
- Leadership development, career advising, soft skills and well-being programs
- Unlimited access to EPAM's internal learning database
- Free English classes with certified teachers
We cover it all:
- Comprehensive medical & family care package
- Seven trust days per year (sick leave without a medical certificate)
- Discounts from 800+ partners (sports activities, restaurants, stores and services)
We connect like-minded people
- Delivering innovative solutions to industry leaders, making a global impact
- Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
- Opportunity to work abroad for up to two months per year
- Relocation opportunities within our offices in 55+ countries
- Corporate and social events
We invest in your growth
- Leadership development, career advising, soft skills and well-being programs
- Certifications, including GCP, Azure and AWS
- Unlimited access to EPAM's internal learning database
- Free English classes with certified teachers
We cover it all
- Participation in the Employee Stock Purchase Plan
- Monetary bonuses for engaging in the referral program
- Comprehensive medical & family care package
- Five trust days per year (sick leave without a medical certificate)
- Benefits package (sports activities, a variety of stores and services)
We connect like-minded people:
- Delivering innovative solutions to industry leaders, making a global impact
- Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
- Opportunity to work abroad for up to two months per year
- Relocation opportunities within our offices in 55+ countries
- Corporate and social events
We invest in your growth:
- Leadership development, career advising, soft skills and well-being programs
- Certifications, including GCP, Azure and AWS
- Unlimited access to EPAM's internal learning database
- Free English classes with certified teachers
- Discounts in local language schools, including online courses for the Kazakh language
We cover it all:
- Participation in the Employee Stock Purchase Plan
- Monetary bonuses for engaging in the referral program
- Medical & family care package
- Six trust days per year (sick leave without a medical certificate)
- Coverage of psychology sessions of your choice
- Benefits package (sports activities, a variety of stores and services)
- Housing support program: preferential mortgage access via Otbasy Bank partnership
We connect like-minded people:
- Delivering innovative solutions to industry leaders, making a global impact
- Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
- Opportunity to work abroad for up to two months per year
- Relocation opportunities within our offices in 55+ countries
- Corporate and social events
We invest in your growth:
- Leadership development, career advising, soft skills and well-being programs
- Certifications, including GCP, Azure and AWS
- Unlimited access to EPAM's internal learning database
- Free English classes with certified teachers
We cover it all:
- Monetary bonuses for engaging in the referral program
- Medical & family care package
- Six trust days per year (sick leave without a medical certificate)
- Coverage of psychology sessions of your choice
- Discounts for fitness clubs and sports programs
- Benefits package (sports activities, a variety of stores and services)
We connect like-minded people:
- Delivering innovative solutions to industry leaders, making a global impact
- Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
- Opportunity to work abroad for up to two months per year
- Relocation opportunities within our offices in 55+ countries
- Corporate and social events
We invest in your growth:
- Leadership development, career advising, soft skills and well-being programs
- Certifications, including GCP, Azure and AWS
- Unlimited access to EPAM's internal learning database
- Free English classes with certified teachers
Discounts in local language schools, including offline courses for the Uzbek language
We cover it all:
- Monetary bonuses for engaging in the referral program
- Medical & family care package
- Four trust days per year (sick leave without a medical certificate)
- Discounts for fitness clubs, dance schools and sports programs
- Benefits package (sports activities, a variety of stores and services)
Skills
Sign in to read and join the discussion.