Technical Program Manager, Inference

Anthropic's Inference organization serves Claude to millions of users and enterprise customers across its own products and the world's largest cloud platforms. The role is to drive the programs that scale the inference stack, which spans accelerator kernels, an in-house inference engine, distributed routing, autoscaling, and capacity management, across GPUs, TPUs, and Trainium. The work covers scaling and infrastructure, velocity, the inference engine, and performance. The role is described as not a status-tracking role: the TPM owns outcomes end to end and works across Research, Safeguards, Security, Compute, Infrastructure, API, and cloud and silicon partners.

Responsibilities

Own the programs that scale Claude's serving fleet across accelerator and cloud platforms, from early direction through delivery; Drive new capacity online end to end, working across compute, infrastructure, security, networking, and inference teams; Build visibility into supply, demand, and utilization with capacity and finance partners, and drive trade-offs among utilization, cost, latency, and reliability to a clear decision; Lead reliability programs and large migrations without disrupting production traffic or research, and make sure incident follow-ups land in team roadmaps; Drive the programs that make inference engineers faster, including productivity metrics, bottlenecks, and how software is built, tested, and released across platforms; Coordinate roadmaps and priorities across the production, research, and safeguards needs that depend on the same teams; Track performance and efficiency work from opportunity to measured impact in production, with correctness held as a release criterion; Own inference readiness for model launches alongside launch TPMs

Requirements

7+ years of experience as a TPM, or in an equivalent engineering or SRE lead role, owning large multi-team infrastructure programs end to end in production distributed systems, ML infrastructure, or cloud platforms; Technical fluency in large-scale serving infrastructure (request routing and load balancing, autoscaling, capacity planning, cluster orchestration, deployment and rollout, observability); A working understanding of how LLM inference uses hardware (prefill and decode, batching, caching, compute versus memory bandwidth versus interconnect), or a track record of getting deep in an unfamiliar technical domain quickly; A track record of building program mechanisms from scratch on ambiguous, fast-moving efforts, and of retiring process that has stopped earning its keep; The ability to drive outcomes across organizations you don't manage, with competing priorities and unclear decision ownership; Strong written communication, including plans, reviews, and executive updates; Low ego and high ownership; Care about Anthropic's mission and about serving frontier AI safely and reliably; Bachelor's degree or an equivalent combination of education, training, and/or experience

Nice to have

Direct experience with LLM or ML inference serving at scale (model servers, batching and scheduling, KV caching, quantization, speculative decoding, multi-model routing); Experience with accelerator infrastructure (GPU, TPU, or Trainium), including cluster bring-up, hardware qualification, or running one workload across more than one accelerator platform; Experience managing capacity across multiple clouds or regions, including demand forecasting, headroom policy, cost modeling, or unit economics for compute-heavy services; Experience with developer productivity, release engineering, or CI/CD at scale, especially for hardware-dependent workloads; Experience running performance or efficiency programs, with grounding in rooflines, kernels, ML compilers (XLA, Triton, Neuron), numerics, or collective communication; Familiarity with Kubernetes-based orchestration and job scheduling; Experience working with research teams and turning their needs into infrastructure requirements; Practical experience using AI tools in your own workflow, including building automations or agents

Benefits

Competitive compensation and benefits; Optional equity donation matching; Generous vacation and parental leave; Flexible working hours; Office space for collaboration; Visa sponsorship available for some roles