Staff Software Engineer Observability

You will own the observability platform's architecture and roadmap across metrics, logs, traces, and events. You will build telemetry pipelines for high-scale infrastructure, define SLOs and alerting strategies, and create tooling for root-cause analysis. You will also establish OpenTelemetry standards and mentor senior engineers.

Responsibilities

• Own the long-term architecture and roadmap for the observability platform
• Drive observability adoption through RFCs, design reviews, and stakeholder partnerships
• Design telemetry pipelines for high-cardinality, high-frequency data
• Solve telemetry ingestion and query-performance problems
• Define SLOs and alerting strategies with engineering, SRE, hardware, and ML infrastructure teams
• Create tooling for root-cause analysis and reduced MTTR
• Surface hardware health signals in a unified observability layer
• Design and implement OpenTelemetry instrumentation libraries and standards
• Establish observability conventions across teams
• Mentor senior engineers and improve engineering practices

Requirements

• 8+ years of software engineering experience
• 4+ years building or operating observability or monitoring platforms at significant scale
• Expertise with open-source observability systems such as Prometheus, Thanos, Cortex, Mimir, Elasticsearch, ClickHouse, or Loki
• Experience with OpenTelemetry in polyglot service environments
• Go proficiency
• Strong Python experience
• Distributed-systems and Kubernetes expertise
• Experience with observability cost management and capacity planning
• Experience setting technical direction and driving adoption across teams