Site Reliability Engineer
Posted Updated
Description
- As a Site Reliability Engineer at Baseten, you’ll define and codify the gold standards of day 2 operations for our ML infrastructure platform
- You’ll envision and build robust systems, processes, automations, and observability tooling that keep our platform reliable at scale — and that empower the broader organization to operate confidently
- You’ll work closely with engineering, forward-deployed and product teams: learning from recurring failure patterns, turning tribal knowledge into automated mitigations, and raising the operational floor for the entire company
- You’ll work on projects like these as part of the SRE team:
- Improve Baseten SRE Practices, by instrumenting SLOs and SLIs, improving alerting and observability for all services
- Building AI-assisted tooling for incident triage and response
- Own the reliability of Baseten’s multi-cloud Kubernetes infrastructure, including incident response, post-mortems, and remediation tracking
- Build and maintain observability infrastructure — metrics, logging, dashboards, and alerting — as code
- Author, validate, and improve runbooks for recurring failure patterns, ensuring they’re structured for low-context, safe execution
- Identify high-frequency failure patterns and convert them into automated mitigations or self-healing automations
- Diagnose and resolve runtime issues related to latency, memory behavior, GPU utilization, concurrency, and model lifecycle management
- Define and instrument SLOs and SLIs across customer workloads and internal services
- Navigate ambiguity, make principled tradeoffs, and avoid unnecessary complexity in the systems you build and the processes you define
Benefits
- Remote-first work environment. The Baseten team is welcome to work from wherever they want; fully remote, in our San Francisco office, or a mix of both. Today, our team (including our founding team) is spread across the United States, Canada, and Armenia. We provide a $1,000 stipend for you to make your home-office comfortable and productive
- Regular in-person team summits. We get together as a team three times a year to plan, workshop, and most importantly, get to know each other better
- Unlimited PTO. We ask that everyone take at least 4 weeks of vacation. And we have a company-wide break between Christmas and New Year’s Day
- Full healthcare coverage. Medical, dental and vision insurance for you and your family
- Paid parental leave. 16-weeks fully paid parental leave (adoptive and non-birth parents included) and flexibility with schedules while returning to work
- Company-sponsored 401(k) for you to contribute to
- Learning and development budget. We encourage you to take classes, attend conferences, and invest in your craft and we’ll cover expenses to make it happen
- Familiarity with incident management platforms (incident.io or similar) is a plus
- Strong foundation in observability tooling: metrics (VictoriaMetrics, Prometheus), logging (Loki, ELK), dashboards (Grafana), and alerting pipelines. Observability-as-code experience is a plus
- Extensive hands-on experience with Kubernetes (multi-cloud experience across EKS, GKE, or similar is a strong plus)
- Comfort working at the intersection of engineering and operations — you write code, but you also think deeply about process, escalation paths, and operational leverage
- Experience in building and maintaining scalable infrastructure
- Experience with infrastructure-as-code (Terraform, Helm) and GitOps workflows (Flux CD, ArgoCD)
- Experience writing and improving runbooks, leading incident response, and doing post-mortem analysis
- No prior ML experience required, but curiosity about how ML models are deployed and served at scale will serve you well
Skills
Sign in to read and join the discussion.