Software Engineer, Cloud

You'll build Ollama’s cloud, a scalable inference platform that lets developers run large, capable open models in their workflow. You'll work on high-throughput, low-latency distributed systems — inference serving, GPU fleet management, routing, metering, and the platform that Pro, Max, Team, and Enterprise customers rely on to process trillions of tokens.

Responsibilities

Build and scale the inference platform that serves every request from ollama.com; Design the routing and capacity layer that places workloads across GPUs and regions for cost, latency, and availability; Own multi-tenant infrastructure: isolation, quotas, usage metering, billing, and Pro/Max/team/enterprise tiering; Build the reliability, observability, and cost controls for our team and customers

Requirements

Deep experience with high-throughput, low-latency distributed systems — inference serving, traffic routing, real-time data pipelines, or large-scale APIs; Comfortable with cost/performance tradeoffs at scale and have owned a production service end-to-end; Experience with Kubernetes, GPU scheduling, or inference infrastructure; Think in terms of reliability, SLOs, and honest capacity planning

Nice to have

experience building an inference platform; GPU fleet management; billing/metering for an AI service