Senior Software Engineer - Distributed Systems

We build and operate observability infrastructure for metrics ingestion and alerting across Workday's global production environments. You will work on distributed systems at scale, collecting, processing, and acting on operational data, with a focus on reliability, performance, and simplicity. You will also contribute to agentic engineering, building tools and workflows that help engineers and AI agents operate the platform more effectively.

Responsibilities

Design, build, and debug distributed systems across cloud-native production environments; Develop platform services for metrics ingestion, processing, and alerting at scale; Shape how AI agents interact with observability infrastructure through tooling, APIs, and operational workflows; Collaborate across teams to deliver infrastructure that is resilient, observable, and easy to operate

Requirements

6+ years of professional software engineering experience with production-grade systems; 4+ years designing, building, and operating large-scale distributed systems on public or private clouds; Strong understanding of distributed systems fundamentals, including concurrency, fault tolerance, high availability, and failure-mode analysis; Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience

Nice to have

Interest or experience building MCP servers, agent-facing APIs, or tooling that helps engineers and AI systems operate complex platforms; Experience with Prometheus-style monitoring ecosystems, time-series metrics pipelines, or large-scale observability platforms in production; Strong proficiency in Go for production services; experience with Python for tooling, automation, data processing, or ML workloads; Experience designing and implementing robust, scalable HTTP and gRPC APIs in multi-tenant environments; Experience deploying and operating services on Kubernetes using infrastructure-as-code and platform configuration workflows such as Helm, Jsonnet, Terraform, or GitOps-style deployment pipelines; Strong testing discipline across unit, integration, and end-to-end tests for distributed systems; Demonstrated ability to own problems end-to-end, work closely with SRE and partner engineering teams, and mentor other engineers; Excellent written communication skills for architecture designs, operational runbooks, and technical documentation

Benefits

Workday Bonus Plan or role-specific commission/bonus; annual refresh stock grants; comprehensive benefits; flexible work schedule (Flex Work)