Software Engineer, Runtime

Ollama is the most popular way for developers to access open models. Our team is small and talent dense. You'll work on the heart of Ollama — the local runtime that runs open models on developers' own machines. It loads models, manages memory, drives GPU acceleration across NVIDIA, AMD, Intel, Qualcomm, and Apple Silicon (including our MLX integration), and makes all of it feel instant. You'll work in Go and C/C++ and touch the model formats and inference engines underneath, shipping to macOS, Linux, and Windows across an enormous range of hardware.

Responsibilities

Make open models run fast and reliably on consumer and enterprise hardware; Own pieces of the runtime: model loading & scheduling memory management, quantization, GPU hardware backends; Integrate new model architectures and quantization formats so the latest open models work on day one; Improve cold-start, time-to-first-token, and throughput; Partner with model labs and hardware vendors on early access and deep integrations; Ship in the open: Ollama is open source, and you'll work with the community

Requirements

Strong systems fundamentals and comfortable in Go, C, or C++; Worked close to the metal — GPU compute, inference, game engines, databases, OS, or networking; Care about performance and have profiled and optimized real workloads; Comfortable shipping to millions of users and handling the long tail of hardware and OS combinations

Nice to have

model quantization; GPU programming (CUDA/Metal/SYCL); Apple MLX