Staff Software Engineer - OpenTelemetry Python instrumentation | Ireland | Remote

Grafana Labs is seeking a Staff Software Engineer to focus on a language-specific implementation of OpenTelemetry in Python, with an emphasis on OpenTelemetry for Generative AI. The role sits within the OpenTelemetry SDKs team, works upstream-first with the OpenTelemetry project, and brings innovations to Grafana Labs products and offerings. The role is remote, with applications considered from Germany, UK, Ireland and Spain.

Responsibilities

Develop and maintain the OpenTelemetry Python SDK and instrumentation libraries, working with maintainers and other vendors on specifications, APIs, and implementation decisions; Build instrumentation that connects agent runs, model calls, and tool execution, including across asynchronous code; Contribute to evolving OpenTelemetry conventions for generative AI, with care for sensitive data and metric cardinality; Use profiling and benchmarks to improve Python SDK performance and make tradeoffs around runtime overhead, compatibility, and maintainability; Be visible as an OpenTelemetry expert by teaching others and promoting best practices within Grafana Labs and the broader OpenTelemetry community; Support the technical direction and vision of the team and contribute to business conversations on OpenTelemetry strategy; Own projects from ambiguous problem through design, implementation, testing, documentation, and release; Partner with GTM, Product and Support teams, join customer conversations, and investigate instrumentation issues

Requirements

8+ years of experience in software development; Strong proficiency in Python; Experience debugging production software, language internals, libraries, and asynchronous code; An interest in AI applications and agent systems; Experience working in open-source communities; Familiarity with observability tooling such as Grafana, Prometheus, and OpenTelemetry; Experience with cloud-based software or systems engineering; Understanding of production environments and ability to assess operational risks; High agency and good communication skills

Nice to have

Experience in Python instrumentation techniques, profiling, runtime diagnostics, or asynchronous context propagation; Experience in a hands-on technical leadership role, setting technical direction and leading project teams; Experience instrumenting AI frameworks or agent workflows, or working with OpenTelemetry semantic conventions for generative AI; Experience contributing to or maintaining open source projects, with evidence of successful pull requests and community collaboration; Experience working on open source observability projects, ideally in OpenTelemetry or Prometheus; Rust experience for exploring Python SDK performance improvements, including Rust-backed implementations

Benefits

Restricted Stock Units (RSUs); 100% remote work; In-person onboarding; Global annual leave policy of 30 days per annum, including 3 Grafana Shutdown Days; Defined career growth pathways; Dedicated individual contributor career path