Senior Staff Machine Learning Engineer (DevOps)
Posted Updated
Description
- The AI Services Platform Group at ServiceNow is a customer-focused innovation group building intelligent software and smart user experiences using existing and latest advanced technologies to enable end-to-end, industry-leading work experiences for customers. We are a group of researchers, applied scientists, engineers, and product managers with a dual mission
- We build and evolve the AI platform, and partner with teams to build products and end-to-end AI-powered work experiences. In equal measure, we lay the foundations, research, experiment, and de-risk AI technologies that unlock new work experiences in the future
- We are seeking a Senior Staff Machine Learning Engineer (DevOps) to lead the architecture, optimization, and operational excellence of our AI inference platform
- You will own the end-to-end inference lifecycle—from onboarding cutting-edge models to deploying optimized inference engines in production across multiple backends on Kubernetes infrastructure
- You’ll also architect the intelligent gateway layer using Envoy and ext_proc to enforce access control, provide observability, and enable advanced request routing
- This role combines deep technical expertise in GPU compute, LLM APIs, inference frameworks, containerization, orchestration, and API gateway architecture with a bias toward pragmatic engineering and measurable performance impact
- Deploy, scale, and operate services in production Kubernetes environments, including Helm-based deployments and CI/CD pipelines that make releases repeatable and safe to roll back
- Own observability for what you build — meaningful metrics, useful logs, real tracing, and alerts that fire on customer impact rather than on noise
- Debug and resolve production incidents independently, participate in on-call, run root-cause analysis, and operate against defined service level objectives
Benefits
- Generous family leave
- Matched donations
- Annual learning stipends
- Flexible PTO
- Competitive retirement plan
- Paid volunteer time
- Expertise in coding efficient, object-oriented, modularized and quality software
- Experience building new products that use challenging algorithms
- Knowledge of core AI/ML techniques and algorithms
- Expertise in Go or Python, OOP, Design Patterns, time and space-efficient algorithms
- Knowledge of unit testing, profiling, and code tuning
- AI-assisted debugging and trace analysis, and a clear understanding of the data privacy rules governing what goes into a prompt
- Confident use of AI coding assistants such as GitHub Copilot or Cursor, paired with the critical eye to review AI-generated code as rigorously as any other — catching logic errors, security anti-patterns, and missed edge cases rather than treating model output as production-ready
- Artificial intelligence strategy is a critical skill at Experienced proficiency for IC5 — it applies to how you build and to what you build
- 3+ years of Kubernetes in production, including Helm and CI/CD
- 8+ years of Backend software engineering experience
- 4+ years of designing and operating distributed systems
- A master’s degree may offset up to one year of the experience minimum. Equivalent practical experience is considered where technical depth is clearly demonstrable
- Bachelor’s degree in computer science, software engineering, or a closely related technical field required
Skills
Sign in to read and join the discussion.