Staff Software Engineer - OpenTelemetry Python instrumentation | UK | Remote

In this role, you will focus on a language-specific implementation of OpenTelemetry in Python, with an emphasis on OpenTelemetry for Generative AI. You will get to work in an upstream-first approach, helping accelerate the OpenTelemetry project while bringing the innovations to Grafana labs products and offerings.

Responsibilities

Develop and maintain the OpenTelemetry Python SDK and instrumentation libraries; Collaborate with maintainers and other vendors on specifications, APIs, and implementation decisions; Build Python instrumentation for AI agents, connecting agent runs, model calls, and tool execution, including across asynchronous code; Contribute to evolving OpenTelemetry conventions for generative AI; Improve Python SDK performance using profiling and benchmarks; Be visible as an OpenTelemetry expert by teaching others and promoting best practices; Support the technical direction and vision of the team and contribute to OpenTelemetry strategy discussions; Own projects from ambiguous problem through design, implementation, testing, documentation, and release; Lend knowledge to GTM, Product and Support teams, join customer conversations, and investigate instrumentation issues

Requirements

8+ years of experience in software development; Strong proficiency in Python; Experience debugging production software, language internals, libraries, and asynchronous code; Interest in AI applications and agent systems; Experience working in open-source communities; Familiarity with observability tooling such as Grafana, Prometheus, and OpenTelemetry; Experience with cloud-based software or systems engineering; Understanding of production environments and ability to assess operational risks; High agency and good communication skills

Nice to have

Experience working on open source observability projects, ideally in OpenTelemetry or Prometheus; Experience in Python instrumentation techniques, profiling, runtime diagnostics, or asynchronous context propagation; Experience in a hands-on technical leadership role, setting technical direction and leading project teams; Experience instrumenting AI frameworks or agent workflows, or working with OpenTelemetry semantic conventions for generative AI; Experience contributing to or maintaining Open Source projects, with evidence of successful pull requests and community collaboration; Rust experience

Benefits

100% remote, global culture; Scaling organization with meaningful work; Transparent communication and regular company-wide updates; Innovation-driven environment with autonomy and support; Open source roots; Empowered teams with high trust and low ego culture; Defined career growth pathways; Approachable leadership; Passionate people; In-person onboarding; 30 days annual leave per annum including 3 Grafana Shutdown Days; Restricted Stock Units (RSUs)