Senior Machine Learning Engineer (Runtime and Serving)
Posted Updated
Description
- The ML Optimization team at Waymo provides a set of tools to support and automate the lifecycle of the machine learning workflow, including feature and experiment management, model development, optimization and monitoring
- These efforts have resulted in making machine learning more accessible to teams at Waymo, including Perception, Planner, Research and Simulation
- We are looking for engineers with ML software & systems expertise to help build the next generation Waymo onboard ML inference engine for Waymo fundamental model
- You’ll work across the entire ML stack from the system perspective, from efficient deep learning models, model compression, ML software (e.g. JAX, XLA, Triton, and CUDA), to . You will be pleasantly challenged with deploying Waymo ML models on limited computation resources
- In this hybrid role, you will report to the Senior Manager of Runtime and Optimization
- Architect and develop an efficient, high-performance ML runtime and serving system tailored for both onboard autonomous vehicle compute and large-scale, offboard data center environments
- Lead the integration and feature development for ML inference runtimes across both domains, balancing the strict real-time latency and memory constraints of onboard systems with the high-throughput, highly concurrent demands of offboard serving fleets
- Drive the strategic migration of ML workloads toward a JAX-native runtime architecture, which includes extending and modifying underlying ML compilers and runtimes (e.g., OpenXLA/PjRT, TensorRT)
- Collaborate with world-class Waymo ML practitioners across perception, planner, and research to analyze system-level ML workloads and apply hardware-aware compute optimizations
- Design and build robust tooling for profiling, benchmarking, and identifying system-level bottlenecks across the end-to-end ML software stack
Benefits
- Medical, dental, and vision insurance for employees and dependents
- Employee assistance programs focused on mental health
- Personalized workplace adjustments for diverse needs and abilities, including physical, mental, and neurodivergent considerations
- Access to mental health apps
- Onsite wellness centers
- Second medical opinion for you and your loved ones
- Medical advocacy program for transgender employees
- Counseling services
- Support programs including menopause benefit
- Competitive compensation
- Regular bonus and equity performance grant opportunities
- Generous 401(k) and regional retirement plans
- Annual cross-company compensation review and pay equity analysis
- 1-on-1 financial coaching
- Fertility and growing family assistance
- Parental leave and baby bonding leave
- Survivor income benefit
- Backup childcare
- Elder care and support
- Caregiver leave
- Paid time off, including vacation, bereavement, sick leave, parental leave, disability, and holidays
- Jury duty leave
- Military leave
- Hybrid work model with remote work opportunities also available
- Educational reimbursement
- Peer learning and coaching platform
- Donation matching programs
- Volunteer hours
- Employee resource groups
- Internal community groups and local culture clubs
- Inspiring spaces to work, recharge, and collaborate with fellow Waymonauts
- On-site meals and snacks
- Fitness centers, massage programs, and ergonomic support
- On-demand fitness, wellbeing, and cooking classes
- Commuter benefits
- PhD in CS, EE, Deep Learning or a related field
- Experience modifying ML compilers, runtimes, or inference engines (e.g., TensorRT, ONNX Runtime, OpenXLA/PjRT, TVM)
- 5+ years production programming in C++
- Experience optimizing ML software for hardware accelerators (e.g., GPUs, TPUs, custom silicon)
- Experience building low-latency, highly concurrent distributed backend systems
- 5+ years of professional software engineering experience focused on building, scaling, or maintaining ML systems and infrastructure
- B.S. or M.S. in CS, EE, Deep Learning or a related field
- 3+ years of production experience in Python and major deep learning frameworks (e.g., PyTorch, JAX)
- Experience building or scaling LLM serving systems, including expertise in distributed inference and performance optimization (e.g., KV/prefix caching, continuous batching)
- Experience with custom kernel development (e.g., CUDA/CUDA Tile, Triton, JAX/Pallas)
- Experience architecting unified serving APIs and optimizing tensor buffer management (e.g., zero-copy data transfer, shared memory) for complex, multi-model inference pipelines
Skills
Sign in to read and join the discussion.