Senior Machine Learning Engineer, Inference - USA

Inworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. It builds first-party speech models, serves LLMs, and runs the inference behind modular APIs designed for high-volume, realtime workloads. This full-time, in-person role is based in the Mountain View office.

Responsibilities

Take a model from the research team, containerize it, optimize its serving, and ensure it runs reliably in production; Pick a direction and build the map as you go on unclear problems; Prioritize impact and shipping stable engineering over purely theoretical optimizations; Question architectures to find better solutions to core latency or throughput problems

Requirements

Ability to learn fast, thrive in ambiguity, and show what they have built, broken, and understood; PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems

Nice to have

Inference Optimization (deep understanding of serving frameworks such as vLLM or TRT-LLM); Model Acceleration (quantization, distillation, caching strategies, continuous batching, paged attention, speculative decoding); High-Performance Systems (C++, CUDA, Rust, or highly optimized Python; profiling and GPU performance tuning); Distributed Systems & Scaling (Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, thousands of concurrent connections); Public work (non-trivial systems projects, open-source contributions to inference engines, deep-dive technical write-ups)

Benefits

Base salary plus bonus and equity; Benefits; Relocation assistance