Software Development Engineer
Posted Updated
Description
- We are an infrastructure engineering team within Workday’s observability organisation. Engineering teams worldwide rely on our platform to monitor and operate their production systems
- We own metrics ingestion and alerting infrastructure, from the pipelines that collect and process metrics to the platform services that detect issues and deliver notifications
- We build distributed systems at scale across Workday’s global production environments, with a focus on reliability and long-term operability
- We build tooling and agent-facing interfaces that help engineers and AI agents understand and operate the platform, and we use them in our own day-to-day work
- We work closely with partner teams across engineering and site reliability, with engineers taking end-to-end ownership of the problems they solve
- We build and operate observability infrastructure for metrics ingestion and alerting across Workday’s global production environments
- You will work on distributed systems at scale, collecting, processing, and acting on operational data, with a focus on reliability, performance, and simplicity
- You will also contribute to agentic engineering, building tools and workflows that help engineers and AI agents operate the platform more effectively
- Design, build, and debug distributed systems across cloud-native production environments
- Develop platform services for metrics ingestion, processing, and alerting at scale
- Shape how AI agents interact with observability infrastructure through tooling, APIs, and operational workflows
- Collaborate across teams to deliver infrastructure that is resilient, observable, and easy to operate
- Strong understanding of distributed systems fundamentals, including concurrency, fault tolerance, high availability, and failure-mode analysis
- 4+ years of professional software engineering experience with production-grade systems
- Demonstrated ability to own problems end-to-end, work closely with SRE and partner engineering teams, and mentor other engineers
- Strong proficiency in Go for production services
- 3+ years designing, building, and operating large-scale distributed systems on public or private clouds
- Strong testing discipline across unit, integration, and end-to-end tests for distributed systems
- Experience designing and implementing robust, scalable HTTP and gRPC APIs in multi-tenant environments
- Experience deploying and operating services on Kubernetes using infrastructure-as-code and platform configuration workflows such as Helm, Jsonnet, Terraform, or GitOps-style deployment pipelines
- Experience with Prometheus-style monitoring ecosystems, time-series metrics pipelines, or large-scale observability platforms in production
- Interest or experience building MCP servers, agent-facing APIs, or tooling that helps engineers and AI systems operate complex platforms
- Experience with Python for tooling, automation, data processing, or ML workloads
- Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience
- Excellent written communication skills for architecture designs, operational runbooks, and technical documentation