Skip to content

CoreWeave

Software Engineer (Inference)

Posted Updated

Description

  • As a Junior Software Engineer (IC1/IC2) on the Inference team, you’ll own a single production service end-to-end shipping features, maintaining dashboards/alarms, and improving operational excellence
  • You’ll collaborate with peers across runtime, scheduling, and networking to move key SLIs
  • Own one model-serving service end-to-end: design, implementation, tests, deployment, and ops
  • Add and tune observability (dashboards, alerts, traces); automate runbook steps to reduce toil
  • Improve latency/throughput via micro-batching, concurrency tuning, and caching
  • Participate in on-call; lead post-incident action items for your service
  • Review peers’ PRs and contribute design feedback in team reviews
  • Working knowledge of Kubernetes (or Slurm/Ray) and CI/CD pipelines
  • Experience with metrics/alerting (Prometheus/Grafana/OpenTelemetry) and structured logging
  • Basic understanding of GPU inference concepts (KV caching, mixed precision, streaming tokens)
  • Proficiency in Python or Go; familiarity with gRPC/HTTP APIs and containerized deployments
  • 0–3 years of software engineering experience (or equivalent) building cloud/distributed services in production
  • Experience with inference frameworks (Triton, vLLM, TensorRT-LLM, Ray Serve, TorchServe)
  • Hands-on performance profiling/benchmarking (e.g., wrk, vegeta, perf, Nsight)
  • Contributions to reliability practices: SLI/SLOs or postmortem improvements

Skills

Apply

Apply to Software Engineer (Inference) at CoreWeave

Create a free account first, or go straight to the employer.

Free RoleWatch account

  • Get an email when new jobs match your search
  • Save jobs and track every application
  • See how well your CV matches each job

Already have one?

No account needed. Opens the employer's own site in a new tab.

See also

Software Engineering jobs by country — openings, pay and top skills →