Software Engineer (Training/Inference, C++)

• We are building the high-performance inference platform that serves Grok to millions of users every day with lightning speed and perfect reliability
• As a Member of Technical Staff - Inference, you will design and optimize large-scale model serving systems end-to-end
• You will own everything from distributed infrastructure (global KV cache, continuous batching, load balancing, auto-scaling) to deep low-level optimizations (GPU kernels, quantization, speculative decoding, tail latency)
• This is a high-impact role where your work directly determines how fast and reliably users interact with Grok at massive scale
• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache)
• Optimize latency and throughput of model inference under real production workloads
• Build reliable, high-concurrency serving systems that serve billions of users with 100% uptime, 0% error rate, and excellent tail latency
• Benchmark, fine-tune, and accelerate inference engines (including low-level GPU kernel work and code generation)
• Develop custom tools to trace, replay, and fix issues across the full stack — from orchestration down to GPU kernels
• Create robust CI/CD infrastructure for seamless endpoint deployment, image publishing, and inference engine updates
• Accelerate research on scaling test-time compute, RL rollout, and model-hardware co-design for next-generation systems

Benefits

• Health and wellness: Comprehensive health insurance including medical, dental, vision, and disability coverage
• Life and family: Life and AD&D insurance and fertility benefits to ensure our team’s well-being and peace of mind
• Flexible vacation: We work hard but avoid burn out. Take time off when you need it
• Visa sponsorship: We support international talent with visa sponsorship to join our team
• 401(k) plan: Retirement savings plan to secure your financial future

• Deep low-level systems programming (C/C++ or Rust)
• Experience with large-scale, high-concurrent production serving
• Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.)
• Algorithmic inference optimizations: quantization, speculative decoding, distillation, low-precision numerics
• Experience with testing, benchmarking, and reliability of inference services
• Strong background in system optimizations: batching, caching, load balancing, parallelism
• Low-level inference optimizations: GPU kernels, code generation
• Experience designing and implementing CI/CD infrastructure for inference

• After submitting your application, the team reviews your CV and statement of exceptional work
• If your application passes this stage, you will be invited to a 15-minute interview (“phone interview”) during which a member of our team will ask some basic questions
• If you clear the initial phone interview, you will enter the main process, which consists of four technical interviews:
• Coding assessment in a language of your choice
• Systems hands-on: Demonstrate practical skills in a live problem-solving session
• Project deep-dive: Present your past exceptional work to a small audience
• Meet and greet with the wider team
• Our goal is to finish the main process within one week
• We don’t rely on recruiters for assessments. Every application is reviewed by a member of our technical team
• All interviews will be conducted via Google Meet