Skip to content

CoreWeave

NewBe an early applicant

Senior Deployment Engineer (LLM Inference & Optimization)

Posted Updated

Description

  • We’re looking for a Deployment Engineer who combines hands-on AI engineering, model optimization, and strong customer-facing skills
  • You’ll work directly with customers to bring AI models and applications into production, improve their performance, and integrate them with enterprise systems
  • You’ll own technical delivery from discovery and benchmarking through deployment and adoption, working closely with customer stakeholders and our product and engineering teams
  • This role requires someone who enjoys writing code, experimenting with models, diagnosing performance bottlenecks, and helping customers achieve measurable improvements in model quality, latency, throughput, and cost
  • Understand customer requirements: Lead technical discovery to define use cases, workloads, infrastructure constraints, and targets for model quality, latency, throughput, and cost
  • Optimize inference: Apply techniques such as quantization, batching, prefix caching, KV-cache optimization, and parallelism to improve performance while preserving output quality
  • Adapt models to customer needs: Improve results through prompt optimization, retrieval-augmented generation, and fine-tuning where appropriate. Evaluate the impact of each approach
  • Troubleshoot performance: Investigate issues across model execution, GPU utilization, memory consumption, serving infrastructure, and application behavior
  • Lead technical customer engagement: Run workshops, present benchmark findings, explain tradeoffs, and guide customers through deployment and optimization decisions
  • Build reusable capabilities: Create evaluation tools, benchmark harnesses, deployment configurations, and runbooks. Bring recurring customer needs and performance findings back to product and engineering
  • —-What success looks like—-
  • You help customers deploy AI solutions that meet agreed quality and performance targets
  • Your work produces measurable improvements in latency, throughput, GPU utilization, and cost while maintaining model quality
  • Customers understand the tradeoffs, trust your technical guidance, and can operate the solutions you deliver
  • Your findings and tools make future deployments faster and more effective
  • Experience with model-serving frameworks such as vLLM, SGLang, TensorRT-LLM, or similar technologies
  • Experience optimizing embedding models, rerankers, vision models, or multimodal systems
  • Experience improving retrieval quality and evaluating end-to-end RAG applications
  • 5+ years of experience in software engineering, ML engineering, AI infrastructure, or a related hands-on technical role
  • Contributions to open-source model-serving, optimization, or evaluation projects
  • Familiarity with GPU-based cloud environments, containers, and monitoring tools used to operate model deployments
  • Experience delivering enterprise AI implementations or customer-facing technical engagements
  • Experience building evaluations and benchmarks to measure model quality, latency, throughput, and resource efficiency
  • The ability to work directly with customers, communicate technical tradeoffs clearly, and translate business requirements into measurable engineering targets
  • Experience developing APIs and integrating model endpoints with production applications and customer data systems
  • Strong Python programming skills and experience with frameworks such as PyTorch, Hugging Face Transformers, or comparable tools
  • Strong ownership and troubleshooting skills, with the ability to move from an ambiguous problem to a tested solution
  • An understanding of GPU memory constraints, numerical precision, batching, caching, and parallelism, including how these affect inference performance
  • Hands-on experience deploying and optimizing AI models in production, particularly LLMs
  • Experience profiling GPU workloads using CUDA, NVIDIA Nsight, or related tools
  • Experience with parameter-efficient fine-tuning techniques such as LoRA or QLoRA
  • Experience with speculative decoding, multi-GPU inference, or distributed model serving

Skills

What Senior AI Engineering jobs ask for — and how much of it you have →
Apply

Apply to Senior Deployment Engineer (LLM Inference & Optimization) at CoreWeave

Create a free account first, or go straight to the employer.

Free RoleWatch account

  • Get an email when new jobs match your search
  • Save jobs and track every application
  • See how well your CV matches each job

Already have one?

No account needed. Opens the employer's own site in a new tab.

See also

AI Engineering jobs by country — openings, pay and top skills →